DOI: 10.14714/CP87.1444

GeoDataHub @ Auckland

Igor Drecki, University of Auckland | i.drecki@auckland.ac.nz

AN IDEA IS BORN

Over 10 years ago, I received an intriguing email from Brian Marshall, then Subject (and Map) Librarian for Geography and Environmental Science at the University of Auckland Library. He provided me with a link to a digital repository of Polish topographical maps published between 1919 and 1948 by the Military Geographical Institute (Wojskowy Instytut Geograficzny — WIG, ­english.mapywig.org). The website was very simple, perhaps even uninviting to the untrained eye, but the wealth of information it contained was simply staggering! For the first time, I could grasp the breadth and scope of WIG’s production, their humble beginnings during the reconstruction of Poland after World War I, and the road to their exemplary topographic mapping of the 1930s, so readily used and reproduced by the invading armies during World War II. It did not take long for me to envisage a similar development here at the University of Auckland, whereby systematic digitisation of New Zealand maps and charts could provide a window into the cartographic heritage of this country. Furthermore, it could lead to a showcase of conceptual developments in cartography, surveying, photogrammetry, and other mapping, earth, and social science disciplines, highlighting their contribution to the development of the nation. It was easy to imagine that such a resource would also support dynamic, innovative, and leading edge historical and GIScience research, facilitating scientific discovery and knowledge building. The question was: where to start?

My initial contact with Marek Zieliński, the brain behind the WIG portal, quickly revealed countless traps and frustrations with his project. Many map libraries and archives were reluctant to let their maps be scanned, particularly when contact scanners were to be involved. Lack of institutional support for Marek’s work and recognition of his vision were the primary reasons for these setbacks. Although he donated copious amounts of his private time and energy, Marek struggled to get funding, generate community support, and purchase decent equipment. Since then, things have changed for the better and, although the situation is far from ideal, the repository of WIG maps is currently the authority on topographic mapping of the Polish territory during the first half of the 20th century. These early contacts, followed by extensive communication with vendors, digitisation project leaders, map librarians, and the cartographic community in New Zealand, helped to place our plans for a comprehensive map digitisation programme on the right footing from the word go!

To clearly define the scope of our undertaking, the decision was made early to focus on capturing authoritative maps and charts produced by the New Zealand government and its agencies (Figure 1). We believed these cartographic products would be the most useful in research pursuits and provide a quality resource for teaching and learning. This decision also defined our strategy with regard to soliciting the help and assistance of various contributors in making the programme a success. Identification and selection of cartographic material for scanning—ideally in mint or very good condition—was an important consideration, particularly with our adopted “scan once” approach, meant to minimise handling of the original maps while creating faithful digital copies. As far as equipment, we were very fortunate to have available a large format scanner capable of digitising original material in full colour at high resolution.

Figure 1. Authoritative Maps: New Zealand Mapping Service series 24 (NZMS 24) map “Dominion of New Zealand with Mountain Features in Pictorial Relief”; published in 1921 at the scale of “one inch to 25 miles” (1:1,584,000) by New Zealand Lands and Survey Department. This innovative, aesthetically attractive, and authoritative map encapsulates the ever-present spirit of New Zealand cartography and its drive for excellence.

Figure 1. Authoritative Maps: New Zealand Mapping Service series 24 (NZMS 24) map “Dominion of New Zealand with Mountain Features in Pictorial Relief”; published in 1921 at the scale of “one inch to 25 miles” (1:1,584,000) by New Zealand Lands and Survey Department. This innovative, aesthetically attractive, and authoritative map encapsulates the ever-present spirit of New Zealand cartography and its drive for excellence.

These early developments allowed us to map the road ahead. Brian Flaherty, Associate University Librarian (Digital Services), was instrumental not only in recognising the value of digitising and making available the body of New Zealand maps but also in understanding the financial requirements of such an undertaking. Brian’s plan was to secure internal University funding towards a pilot project that would allow us to get off the ground and attract the support of external organisations and agencies to see the continuation of our map digitisation programme into the future, should we successfully deliver the goods. He drafted a proposal to the Vice-Chancellor’s Strategic Development Fund asking for funding towards “a pilot project to develop a repository of cartographic and geospatial materials for the University of Auckland.” This proposal, put forward collaboratively by the School of Environment and the University Library, was subsequently approved and the wheels were set in motion.

THE PILOT PROJECT

The pilot project served two strategic goals: proof of concept and the identification of opportunities and issues related to map digitisation. The proof of concept resulted in the development of a unique expertise in digitisation, image processing, georeferencing, and capturing metadata for cartographic materials, supported by detailed documentation of the processes involved. Most of this work was done by Shannon McColley, Lead Digitisation Technician, who became a digitisation guru and a driving force behind these developments. Shannon coined the pilot project acronym IMAGINZ (Inventory of Maps and Geospatial Imagery of New Zealand) which quickly became a household name amongst the project team and the wider mapping community following her presentations on the subject at local and national fora, especially the GeoCart cartographic conferences. These often highly technical talks provided many tips and tricks on map digitisation, particularly relating to the use of contact scanners and the processing of map images.

Based on the rapid success of IMAGINZ, we were able to attract the attention of additional partners who contributed both expertise and further funding. Ultimately, this led to our building an extensive network of contacts and partnerships with a number of organisations, most notably the National Library of New Zealand (NLNZ), the National School of Surveying at the University of Otago, Land Information New Zealand (LINZ) and the New Zealand Defence Force (NZDF).

NLNZ offered training on physical map handling and identifying the suitability of cartographic material for scanning using our equipment. They provided guidelines and parameters for digitisation (600ppi optical resolution, 24-bit RGB colour depth, Adobe 1998 colour space, uncompressed TIFF format, etc.) and provided extensive quality assessment of our scans to make sure they meet NLNZ preservation requirements. The expertise of the National School of Surveying was highly valuable in the area of georeferencing cartographic materials to maximise their application in GIS analysis. LINZ, the custodian of most of the maps and charts targeted by our project, provided access to their map collection, mostly in excellent condition, for scanning. LINZ’s maps and charts are held at the NZDF Map Library and NZDF graciously hosted our equipment and personnel on site for several years. The knowledge of Michelle Phillips, NZDF Map Librarian, became quickly appreciated when we had to find a particular map amongst over 100,000 others stored in the library or when we needed historical background on various map series earmarked for scanning. Michelle’s assistance and patience in answering our endless questions constantly reminded us about the true collaborative nature of the project, manifested in the wide-ranging institutional support we received and in the genuine help of individuals who, like us, believed in the value of our endeavour.

Between August 2009 and December 2011, IMAGINZ delivered what Brian was hoping for from the pilot, and more! We had researched, tested, and documented the entire process of digitising sheet maps and charts, from handling the originals and mastering the scanning equipment to image manipulation, georeferencing, and metadata capture. Over 3,750 maps published by the New Zealand government had been scanned and made available to the public. Through forging strategic partnerships with relevant organisations, the pilot project provided a platform to expand this growing repository into a nationally significant archive of historical and contemporary cartographic and geospatial materials. The digitised maps also became part of the National Digital Heritage Archive maintained by NLNZ and provided important nomenclature information to the New Zealand Geographic Board, responsible for researching and approving the official geographic names in New Zealand and the Ross Dependency in Antarctica. Lastly, we managed to attract much needed financial support for the continuation of our efforts into the future.

THE DIGITISATION PROGRAMME

In July 2011, the University of Auckland signed a formal, three-year agreement with NLNZ for “the digitisation of, and creation of metadata for, cartographic and geospatial materials for the research, government and educational communities of users, as well as to the public of New Zealand.” This important collaboration, which started in January 2012, was subsequently extended twice and eventually concluded in June 2016. The IMAGINZ team was charged with running the digitisation programme, taking advantage of its experience gained during the pilot project.

The start of the partnership with NLNZ coincided with the appointment of Laura Armstrong to the position of the GIS and Geospatial Librarian. Her task was to consolidate geospatial information distributed around the University into a single repository managed by the Library and to develop an online geospatial data guide about the cartographic and GIS resources available at the University. Laura’s contribution, however, went far beyond her original work portfolio. She was one of the first to realise the value of incorporating the digitised maps and charts into the new centralised repository. Laura’s advocacy in this area resulted in increased digital storage to accommodate map images, standardised geospatial metadata records, and the incorporation of new content into the geospatial data guide, bringing additional exposure to digitised cartographic materials. But perhaps her most significant contribution was to persuade the Library to consolidate the digitisation programme team under one roof. Implemented in two stages in 2014 and 2015, this move considerably improved the management and decision making concerned with staffing and work planning.

The digitisation programme concluded in December 2016 following a 6-month winding down period sponsored by the University Library. During this time, we focused on filling the gaps in the collection, completing the processing of scanned maps and charts, georeferencing, creating geospatial metadata records, and updating and finalising documentation. Since the beginning of scanning in March 2010, we have digitised over 25,000 maps and charts, of which just under 6,500 are now georeferenced. Almost all of these maps were scanned by Shannon—an incredible achievement in its own right!

THE GEODATAHUB

The consolidated repository of cartographic and geospatial resources started by Laura eventually evolved into the GeoDataHub (GDH). GDH aspires to provide an authoritative and freely accessible national repository of historical and contemporary digitised maps, aerial photographs, satellite imagery, and geospatial datasets of New Zealand to research, government, and educational communities of users, as well as to the public of New Zealand. In addition to the scanning of maps and charts, the ongoing GDH objectives include providing integrated and enhanced data access, including an interface for textual, spatial, and temporal searching, facilitating metadata harvest and discovery, and demonstrating the value of the GDH through promoting innovative research utilising the geospatial resources provided (Figure 2). Furthermore, GDH aims to build a knowledge base concerning New Zealand’s cartographic heritage, with particular focus on authoritative mapping.

(Re)Discovery: While working on NZMS 3 NZ Mosaic Map sheet N4/9 & N6/5 from 1953 (1), Benjamin Jones, Digitisation Technician, noticed a small island north of Simmonds Islands (see red circle), which he could not see on the then-current 2013 NZTopo50 map (4). Further research revealed the island was last shown on the imperial NZMS 1 NZ Topographic Map in 1977 (2) before disappearing from the metric NZMS 260 Topographic Map published in 1984 (3) and all subsequent maps for over three decades! His finding was forwarded to LINZ who, after its own investigation, confirmed the island’s existence and reinstated it on the most recent edition of NZTopo50 map (5) in 2016.

Figure 2. (Re)Discovery: While working on NZMS 3 NZ Mosaic Map sheet N4/9 & N6/5 from 1953 (1), Benjamin Jones, Digitisation Technician, noticed a small island north of Simmonds Islands (see red circle), which he could not see on the then-current 2013 NZTopo50 map (4). Further research revealed the island was last shown on the imperial NZMS 1 NZ Topographic Map in 1977 (2) before disappearing from the metric NZMS 260 Topographic Map published in 1984 (3) and all subsequent maps for over three decades! His finding was forwarded to LINZ who, after its own investigation, confirmed the island’s existence and reinstated it on the most recent edition of NZTopo50 map (5) in 2016.

One of the facets of GDH is the web-based Cartographic and Geospatial Resources Repository (gdh.auckland.ac.nz), which launched in August 2016. This developing repository of map images features maps published by the New Zealand Lands and Survey Department (later called Department of Survey and Land Information, and now LINZ) since the 1860s. Digital topographic databases produced by LINZ since 2011 for New Zealand and its offshore islands, the Cook Islands, Niue, Tokelau, and the Ross Dependency in Antarctica are also provided. Over 20,000 maps, charts, and datasets are now available and, as of 31 October 2017, nearly 2,000 users from 50 countries have visited the website to browse and download data. All resources contained in the repository are subject to a Creative Commons Attribution 3.0 New Zealand (CC BY 3.0 NZ) licence, creating ample opportunities to use and reuse the data. This open access is welcomed by many communities, particularly artists, designers, historians, cartographers, developers, and geospatial analysts.

GDH has already enabled a number of valuable and even ground-breaking projects. One of these involved researching the spatial extent of glaciers in the Southern Alps through time by comparing georeferenced imperial “one inch to a mile” (1:63,360) topographic maps dating back to the early 1940s and their metric 1:50,000 equivalents published between the 1970s and 2000s to current topographic datasets and imagery. This auxiliary study contributed to a body of research that delivered a surprising outcome suggesting that over fifty New Zealand glaciers actually advanced over the last 30–40 years: quite the opposite result to a global trend of glacial retreats reported elsewhere.

Another initiative using our digitised maps is MapsPast (mapspast.org.nz), a community-based project providing a visual window into the historical and contemporary geography of New Zealand. Its trademarks are predominantly seamless, country-wide map coverages produced at 10-year intervals from 1899 to 2009. They are created by georeferencing and stitching together national cadastral or topographic map series obtained almost exclusively from GDH. MapsPast has a wide following amongst varied users who find the service very valuable for their work or personal pursuits. Recently, an investigation into improving the national DTM has been undertaken. One approach is to assess the suitability of the 1:25,000 photogrammetric plots for this task (Figure 3). These plots were created between the 1970s and 1990s as an intermediate step in producing the 1:50,000 national topographic map series. They contain a much higher density of spot heights than the published and subsequently vectorised topographic maps, hence supposedly providing better resolution for modelling the third dimension. The study is still ongoing but we are already delighted to see GDH resources being used in this investigation.

Figure 3. Valuable Projects: GDH is a valuable source of information for an ongoing investigation concerned with enhancing the national DTM. The 1:25,000 photogrammetric plot (left) records a far greater density of spot heights (red circles) than the equivalent published 1:50,000 topographic map of the same area (green circles), arguably providing better resolution data for modelling the 3rd dimension.

Figure 3. Valuable Projects: GDH is a valuable source of information for an ongoing investigation concerned with enhancing the national DTM. The 1:25,000 photogrammetric plot (left) records a far greater density of spot heights (red circles) than the equivalent published 1:50,000 topographic map of the same area (green circles), arguably providing better resolution data for modelling the 3rd dimension.

GDH is very much a work in progress. Our current effort concentrates on a web-based catalogue application that can manage our cartographic and geospatial resources and provide powerful search functionality. The next phase will involve adding new content, not only digitised maps and charts but also geospatial datasets and imagery. We are continually working on a knowledge base that will deepen general understanding of our national cartographic heritage and hopefully support discovery and innovation. Our often serendipitous “treasure hunting” and constant pursuit for the lost and forgotten will ensure continued sharing of exciting cartographic material and mapping stories for a long time to come.

THE HEART

This story is foremost a testimony to the vision, support, and determination of the people who were, and often still are, at the heart of our journey. These few mentioned here represent many who share the true spirit of our undertaking and genuinely care for New Zealand’s cartographic heritage. Some, like Marek and Brian (Marshall), came up with great ideas and were happy to provide invaluable insight based on their experiences—good and bad—with similar map digitisation projects. Others, like Brian (Flaherty) and Shannon, made sure we were well resourced, both in terms of personnel and funding, to keep us going forward, ensuring that the volume and quality of our scanning satisfied (or exceeded) both the programme goals and industry standards. Yet others, like Michelle and Laura, willingly contributed their knowledge to assist us in understanding the stories behind the maps and charts published in New Zealand and steered our programme towards new heights by optimising digital storage infrastructure and facilitating a centralised repository of cartographic and geospatial materials augmented by an informative online data guide.

Collectively, this group of dedicated people not only made significant professional and often personal contributions but also successfully advocated for our cause at their workplaces and communities. This in turn resulted in our obtaining wide-ranging institutional support, including the provision of access to quality cartographic collections and financial backing that sustained a digitising programme for many years. In essence, these developments put us on the solid path that has seen the gradual transformation of what was essentially a large map scanning exercise into a growing GDH service. This would never have been possible without so many sharing in the vision and going the extra mile.

We are very fortunate to see the real difference these cartographic and geospatial resources are making to various government, research, and educational communities. Equally validating is how they have been embraced by the general public of New Zealand. Melanie, a GDH user, wrote, “Just wanted to congratulate you (and all involved). This is an awesome project and a great teaching opportunity for students.” We are truly humbled by the appreciation we receive from people like Melanie and find it an inspiration to keep up the good work!