Mar 13, 2024
Version 2
NGGDPP Collection Metadata Submission Guide V.2
- 1USGS
External link: https://webapps.usgs.gov/rescicoll/index.html
Protocol Citation: darthur Arthur 2024. NGGDPP Collection Metadata Submission Guide. protocols.io https://dx.doi.org/10.17504/protocols.io.6qpvr321bvmk/v2Version created by darthur Arthur
License: This is an open access protocol distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Protocol status: Working
We use this protocol and it's working
Created: February 22, 2024
Last Modified: March 13, 2024
Protocol Integer ID: 95606
Keywords: ReSciColl, collections, scientific collections, metadata, catalog
Disclaimer
Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the U.S. Government.
Abstract
The Registry of Scientific Collections (ReSciColl) is a metadata catalog administered by the National Geological and Geophysical Data Preservation Program (NGGDPP). The catalog includes essential metadata describing scientific collections, which promotes discovery and encourages further exploration of similar items by identifying contacts, providing access instructions, and linking to online resources for additional information.
This submission guide describes the process by which collection and item metadata is to be submitted to ReSciColl. It introduces mdEditor, an open-source, browser-based application for creating metadata records that is highly customizable and uses a graphical user interface developed by the Alaska Data Integration Working Group (ADIwg), a collaboration between the U.S. Geological Survey and the U.S. Fish and Wildlife Service. Many of the mdEditor screenshots used to illustrate this document are from the ADIwg mdEditor online tutorial and reference manual, which are available at https://guide.mdeditor.org/. ReSciColl users are highly encouraged to familiarize themselves with the online tutorial, the reference manual, and this submission guide, which are all valuable resources for navigating mdEditor, creating metadata, and publishing it to ReSciColl.
Guidelines
Contact Information
National Geological and Geophysical Data Preservation Program
Dan Arthur, NGGDPP Interdisciplinary Scientist, U.S. Geological Survey
Mikki Johnson, NGGDPP Associate Program Coordinator, U.S. Geological Survey
Lindsay Powers, NGGDPP Program Coordinator, U.S. Geological Survey
Materials
Introduction to ReSciColl and mdEditor
ReSciColl, the NGGDPP Registry of Scientific Collections, is a metadata catalog for making information about scientific collections Findable, Accessible, Interoperable, and Reusable (FAIR). Metadata in ReSciColl are organized in a three-tiered hierarchy by organization, collection, and items within the collections. Organization information and metadata for items within a collection are managed directly within ReSciColl, while collection metadata are first created and exported from an associated tool called mdEditor.
This submission guide is organized into sections and steps which illustrate the process of creating metadata for a collection in mdEditor, submitting it to ReSciColl, and then formatting and submitting metadata for items within a collection.
mdEditor (https://go.mdeditor.org/) is a web application that allows flexibility in the creation of metadata records for various purposes. The NGGDPP has supported many recent developments in mdEditor to meet the needs of ReSciColl. Further information about mdEditor, including a detailed tutorial and user manual, along with information on other related and connected tools, can be found at https://www.mdtoolkit.org/.
Important: Even though mdEditor is accessed via web browser, it keeps all settings, records, and other information locally, in the browser cache, until either exported or published (both are available functions in the application). A best practice is to export (save) records regularly, so that local copies are available and can be imported again for additional editing, publication, or updates, in case the browser cache is cleared. Multiple users can edit a copy of the same record by exporting and importing the content, since there is no way to access an unpublished record except through mdEditor via the web.
Exporting and importing will be covered in their respective sections of this guide, and additional information can be found in the tutorial and reference manual at https://www.mdtoolkit.org/.
If you are using mdEditor for the first time, you may be presented with the warning popup like that shown in Fig. 1. It may also appear after mdEditor has been updated by the developers.
Figure 1: mdEditor update alert pop-up.
Click “OK” to continue. A clean instance of the mdEditor dashboard is shown in Fig. 2. mdEditor will show up blank when you first open or after a cache clear as it does not connect to existing ReSciColl collections or content. Fig. 3 depicts an instance of mdEditor with several records of each type. The left side of the mdEditor window lists metadata records, contacts, and dictionaries currently available. Each section can be expanded/collapsed via the carat icon (^) to the left of the section title in its title bar. The main part of the dashboard also shows the three record types with a count of records available by type, 1) metadata, 2) contacts, and 3) dictionaries.
Figure 2: mdEditor clean dashboard.
Figure 3: mdEditor dashboard with records and callout boxes highlighting features.
Before start
Considerations before Starting
To begin, determine whether you are starting a new collection, modifying an existing collection, or modifying an organization's suite of collections.
1. New collection – Follow the process in this guide, beginning with Step 1. If you have existing collections and/or contacts and are new to mdEditor, request the ReSciColl migration files for your organization (referred to under item #3 below), if applicable.
2. Modify a single existing collection – If contacts need to be updated, start with the best contacts migration file for your organization (refer to item #3 below for information on how to request). Then, refer to the Import section, followed by the section, Prepare for ReSciColl Ingest.
3. Modify an organization's suite of collections – Make updates to a suite of existing collections to update ReSciColl. Contact nggdpp@usgs.gov for the ReSciColl metadata files that include the contacts and existing collections metadata files for your organization. Follow the guidance in the sections of this guide that describe the various tabs of mdEditor, work through the collections you wish to update, and proceed with the Prepare for ReSciColl ingest section.
If you have existing collections to update with thousands of items and/or file attachments in a collection, contact nggdpp@usgs.gov for consultation and assistance with management of the items in the collection.
Contacts - Importing
Contacts - Importing
In mdEditor, Contacts are created and maintained as separate records from metadata records. This enables a contact to be used many times in a single metadata record or across multiple metadata records without requiring a user to reenter the contact's information. Rather than entering a contact's information within a metadata record, mdEditor provides users with a drop-down list of contacts that can be linked to various roles within the metadata record. The available list consists of those contact records currently in the user’s browser instance of mdEditor, typically entered or imported by the user. Like metadata records themselves, contacts can also be imported and exported via JSON format. The NGGDPP is working with ReSciColl users to develop standardized contacts that can be imported and shared among other ReSciColl and mdEditor users within the same organization.
Refer to the video below for a walk through the mdEditor Import function (for a version of the video with supported closed captions and a transcript of the audio, see the USGS NGGDPP Data Submission page at this link: https://www.usgs.gov/programs/national-geological-and-geophysical-data-preservation-program/data-submission):
To import Contacts, select the “Import” option on the top menu bar in mdEditor. The import selection screen will appear (Fig. 4).
Figure 4: mdEditor Import Data screen with callout boxes highlighting features.
If the file you wish to import is local (on the user's PC or connected storage), click the large blue box. If it is at a remote web address, enter the address in the “Import from Online URL” box. mdEditor supports three file formats for import: mdEditor (.json), mdJSON (.json), and FGDC CSDGM (.xml). Contacts will generally be shared via the mdEditor JSON format, which can also hold metadata records in the same file. Once the file is selected, the Import Data selection list screen will appear (Fig. 5).
Figure 5: mdEditor Import Data Records/Contacts selection list.
In the example in Fig. 5, Contacts are included below a set of Metadata Records. The Contacts available for import are shown in Fig. 6.
Figure 6: mdEditor Import Data list, showing Contacts available for import.
All records and contacts are selected by default. Deselect any you do not wish to import, select “Merge” on the “Import/Merge” toggle on the upper right (Fig. 5), and select the “Click to Import Data” button. Fig. 7 shows a previously clean instance of mdEditor with all the example records and contacts from Figs. 5 and 6 imported (in the lists on the left side of the screen).
Note: "Replace" will replace any metadata records and/or contacts currently loaded. "Merge" will add new metadata records and/or contacts to those currently loaded. It will also update with the imported records and/or contacts any currently loaded records and/or contacts that have the same Universally Unique Identifier (UUID; an internally generated code used by mdEditor to identify unique records within the application).
Figure 7: mdEditor Dashboard after importing 9 Metadata Records and 9 Contacts.
Contacts - Creating and Editing
Contacts - Creating and Editing
If you need to create a new contact record in mdEditor, click the plus sign (+) to the right of the “Contacts” section header on the left side (refer to the “Create New Contact” callout in Fig. 9). As shown in Fig. 8, you can choose whether to create a contact for an individual (e.g., Jane Doe, Richard Roe) or an organization (e.g., U.S. Geological Survey, U.S. Department of Interior). Clicking the switch icon between the labels for Individual and Organization will toggle the type of contact to be created. The field labeled “Contact ID” is a system generated Universally Unique Identifier (UUID) for the contact. It is used by mdEditor to link the contact to other contacts and/or metadata records. If your organization uses a different identification code system for identifying contacts, you will need to change the value here from the generated UUID to your organization’s ID at this point, before proceeding. Enter either the individual’s name or the organization’s name in the “Name” field as appropriate, and, if creating an individual contact, a “Position Name” (e.g., Curator) is also required. Required fields in mdEditor are always indicated by a red asterisk (*) after the field name.
Figure 8: Create New Contact dialog.
Click “Save,” and the full contact edit window will be presented (Fig. 9).
Figure 9: Editing Contact window, with callout boxes highlighting features.
In the contact editing window, you can add all relevant metadata for the contact, including phone numbers, email addresses, etc. You can also link an individual or organization to another organization via the “Member of Organization” drop-down list, presuming the contact for the desired organization has already been created or imported.
If your organization has more than one user creating and/or maintaining metadata records for ReSciColl, you may wish to share contact records in order to maintain consistency and avoid creating duplicate contacts for an individual or organization. If three people in an organization each created a contact for John Smith but used different spelling (e.g., two people create a contact using "John Smith," and a third creates one for the same person, but using "John A. Smith,"), each contact would be considered a different individual and would have a unique UUID/Contact ID, even the ones with identical names. Exporting contacts from mdEditor allows sharing contact records in order to minimize the likelihood of duplicate contacts. Exporting contacts will be covered in the Export section of this guide, along with general exporting from mdEditor. When creating or editing contacts, including persistent identifiers such as ORCiDs and/or RORs is highly encouraged as this practice increases the accuracy and quality of the metadata.
Additional information on Contacts in mdEditor can be found in the online tutorial and reference manual:
Creating a Collection Record for ReSciColl Using mdEditor
Creating a Collection Record for ReSciColl Using mdEditor
ReSciColl users who have collection or item metadata must first create the collection metadata record before uploading metadata for the items within a collection. To add a collection in ReSciColl, visit the ReSciColl website:
In the “Collections Management” section on the lower right side of the page, under "Create and Update Collections," click on the “Create and Update Collections in mdEditor” button (highlighted by the yellow ellipse in Fig. 10):
On the mdEditor dashboard, in the primary sidebar on the left side, click the plus (+) sign to the right of the count of metadata records to create a new record (Fig. 11).
Figure 11: mdEditor dashboard with callout box highlighting where to add new record.
The “Create New Record” screen appears (Fig. 12).
Figure 12: mdEditor Create New Record 1st screen with callout boxes highlighting features.
As indicated by the callout boxes in Fig. 12, you will need to provide a title for the record. This is the collection title that will be displayed by ReSciColl. In the Resource Types section, from the Type drop-down list, choose the appropriate resource type(s). Important: For ReSciColl, there must always be at least one Resource Type with a value of “collection.” In the Name field, you can provide a name for the type of collection to be represented by this record (e.g., field notebooks, hyperspectral image data, thin sections).
Once you have entered a title and resource type, the “Save” button will become available.
After saving the initial information for the record, as shown in Fig. 13, the main edit window for mdEditor will appear. The screenshot in Fig. 13 shows the various sections of the mdEditor interface when in edit mode. Refer to the online tutorial and reference manual for more information: https://guide.mdeditor.org/.
Figure 13: mdEditor Editing Metadata Record screen, showing the Main tab and callout boxes highlighting features. "Breadcrumbs" show the path taken within mdEditor to get to the current screen, and can be used for navigation within the application.
Before creating a metadata record for a collection with mdEditor, users should ensure they have the appropriate contact records in mdEditor, as discussed in the Contacts section (Steps 1 - 5).
One of the NGGDPP requirements and best practices for ReSciColl, as well as several metadata standards, is to record the people and/or organizations related both to the resource being described by the metadata record and those involved in creating and maintaining the metadata record itself. Contacts relevant to the new metadata records must be created in or imported to your mdEditor Contacts pane before they can be associated with the metadata record.
Profiles
Profiles
mdEditor is designed with the use of what are referred to as profiles to customize the visibility of metadata elements when creating a metadata record (Fig. 14). Profiles may also be considered views, as all metadata elements remain available, but a particular profile, when loaded, may not display every possible field for entry. Profiles may also be used for custom enforcement of required content depending on the type of resource being described by the metadata record. Profiles can only be switched while in edit mode.
For ReSciColl, an initial set of five domain profiles are currently loaded along with the defaults:
1. NGGDPP Common: A core profile which ensures all metadata elements required by the NGGDPP are collected. All elements visible in this profile are also visible in the following four domain-specific profiles.
2. NGGDPP Geological: A profile designed for collections of geological samples.
3. NGGDPP Image: A profile designed for collections of images/photographs.
4. NGGDPP Invertebrate: A profile designed for collections of biological samples of invertebrate species.
5. NGGDPP Vertebrate: A profile designed for collections of biological samples of vertebrate species.
The NGGDPP continues to develop additional profiles for other collection types, and suggestions are always welcome at the NGGDPP email (nggdpp@usgs.gov).
Before continuing with creating a record, select the profile appropriate for the type of collection being described. Changing the current profile can be done in the top menu bar, in black, via the drop-down menu labeled “Profile” (Fig. 14).
Figure 14: mdEditor profiles drop-down menu, featuring NGGDPP domain profiles.
Switching profiles via the drop-down menu will change what metadata elements, sections, and menus are shown, as well as, in the case of the NGGDPP domain profiles, what keyword lists are available. Important: Profiles can always be switched during editing, as the entire metadata schema (all defined fields/elements) is always available and accessible via the "Full" profile. Each of the NGGDPP domain profiles (Geological, Image, Invertebrate, Vertebrate) displays all the fields and keyword lists available in the NGGDPP Common profile, along with specific fields and keyword lists deemed relevant to those domains.
Once you have selected a profile, you are ready to enter the metadata for the collection into the record. The basics of navigating the mdEditor interface are covered in an online tutorial that is available at:
In order to maximize the Findability, Accessibility, Interoperability, and Reusability (the FAIR Principles) of ReSciColl collections, the recommended best practice is to complete as much of the available metadata as possible. When creating a collection record in mdEditor, visit each of the
available tabs for the selected profile, and provide as much information as is available for each field, whether or not a field is required. Also, if you have metadata information for an element that is not visible when using a specific profile, switch to the "Full" profile during editing to find the desired element.
mdEditor comes with four default profiles:
- Full (every available metadata element in the mdJSON schema)
- Basic (minimal subset of metadata elements)
- Product (subset of metadata elements commonly used for product metadata, e.g., data releases)
- Project (subset of metadata elements commonly used for project metadata, e.g., field campaigns)
This guide describes fields in the "NGGDPP Common" profile.
Main Tab
Main Tab
Points of Contact
On the Main tab in mdEditor, immediately below Resource Type, is a section called “Points of Contact” (Fig. 15). The contacts added in this section on this tab should be people and organizations associated with the specific collection resource being described. Important: At least one contact must be added as Point of Contact with a role of “owner.” Other contacts may be added here with other roles (e.g., principalInvestigator, collaborator), including associated organization contacts. The complete list of available roles, along with short definitions, can be found here: http://mdtools.adiwg.org/#codes-page?c=iso_role
Figure 15: mdEditor Main tab, Resource Types and Points of Contacts sections.
In the drop-down list for Role, select “owner,” and then select from the available contacts via the Contacts drop-down list to the right. Your collection metadata record should have at least one owner, and it can have more than one. To add additional owners, continue to select them from the Contacts drop-down list. You can only have one entry for a particular role, but that role can have multiple contacts associated with it. If you have additional contacts in other roles, you can add them via the “Add” button on the upper right of the Points of Contact section.
Citations
Citation sections can be found in several locations in mdEditor, because a metadata record can reference many different resources, including associated resources, documents, taxonomic systems, etc. On the Main tab, the citation is for the resource itself (the collection). A blank Citation section is shown in Fig.16. It inherits the Title from the record title, and other information can be added by clicking either the “Edit” or “Edit Citation” button. Doing so will bring up the “Editing Citation” window, in which several metadata items can be recorded, including responsible parties, online resources, and identifiers. Important: Adding a persistent identifier such as a DOI, if available, to the citation for the collection is highly encouraged, and increases the quality and findability of the metadata.
Figure 16: mdEditor Citations display section (empty).
To add an identifier, click the “Add Identifier” button, which will open the section shown in Fig. 17.
Figure 17: mdEditor Editing Identifier dialog box.
Fig. 17 shows the “Identifier” field in which a DOI has been entered (only the DOI itself is required – the full web address is not required). Selecting “DOI” in the “Namespace” field below the identifier is also required.
Description
Also on the Main tab is a Description section, where a required abstract for the collection must be entered (Fig. 18). mdEditor provides a box that supports Markdown and rich text formatting. As shown in Fig. 18, during editing, mdEditor will highlight items to be checked for spelling. The maximum size of an abstract in mdEditor is 300 characters. A Short Abstract field is also available, for an abbreviated description of the collection.
Note: There is no “Save” option specifically for the edit window for Abstract (or for many other sections in mdEditor). Once you enter or change text in the window, mdEditor recognizes the record has changed, and you can save it via the main “Save” button for the record, or it will autosave periodically if you have that option turned on in Settings (described in the "Settings" section of this protocol, Step 30).
Figure 18: mdEditor Description text editor window for Abstract.
The Description section also provides text entry boxes for “Purpose” (to describe the purpose of the collection) and “Supplemental Information” (any other useful information about the collection that is neither the purpose nor part of the abstract). These fields are not required, but their use is highly encouraged.
Time Period
Also on the Main tab, the Time Period section is used to describe the date range of the collection (Fig. 19). While single dates can be entered in many places in mdEditor, this section allows for a range, when both a Start Date and End Date are known.
Figure 19: mdEditor Time Period section.
Clicking the calendar icon to the right of each date field will bring up a calendar-based date picker (Fig. 20). Times other than 00:00:00 must be entered numerically after the date is selected. Dates and times in mdEditor are local to the user’s instance, to the system clock for the computer being used.
Figure 20: mdEditor date picker calendar popup example.
More information on the Main tab can be found in the online reference manual:
Metadata Tab
Metadata Tab
Basic Information
Note: The "Main" tab discussed in the previous section is used to record information about the collection itself. In contrast, the "Metadata" tab discussed in this section is used to record information about the metadata itself.
Select the status of the metadata via the drop-down list and select the “Add Date” button to associate a date with the record (Fig. 21).
Figure 21: mdEditor Metadata Status section.
You can add a date via the date picker (calendar icon). Important: For “Date Type,” you must have at least one date with a date type of “creation” (Fig. 22), which is the date that the metadata record was created in mdEditor. A description to provide further information about the date is optional. If you are updating an existing record, it should already have a date associated with “creation,” and you should therefore add an additional date entry, e.g., “revision,” or another appropriate Date Type from those available in the drop-down list. Definitions for the available Date Types can be read by hovering the mouse over the question mark for each type in the drop-down list.
Figure 22: mdEditor Dates dialog.
Note: These dates on the mdEditor Metadata tab are dates that are associated with the metadata record itself, not the resource being described (the collection itself). Dates related to the collection are added on the Main tab, which is described in the Main Tab section above.
Metadata Contacts
People and organizations associated with creating, updating, and maintaining the metadata associated with the resource should be added to the Metadata Contacts section on the Metadata tab. At least one contact must be added with a role of “author,” which represents the author of the metadata. To add a metadata contact to a collection metadata record, scroll down until you find the “Add Metadata Contact” button (Fig. 23).
Figure 23: mdEditor Metadata Contacts section (empty).
In the drop-down list for Role, select “author,” and then select from the available contacts via the Contacts drop-down list to the right. Your collection metadata record should have at least one author, and it can have more than one. To add additional authors, continue to select them from the Contacts drop-down list. You can only have one entry for a particular role, but that role can have multiple contacts associated with it. If you have additional metadata contacts in other roles, you can add them via the “Add” button on the upper right of the Metadata Contacts section (Fig. 24).
Figure 24: mdEditor Metadata Contacts section, Role and Contacts selection example.
Metadata Identifier
This section of the Metadata tab is where mdEditor stores the UUID it assigns to the metadata record when you create it. It can be edited if your organization uses a different identification system (as with the UUIDs for Contacts described in that section), but only one metadata identifier object is permitted by the schema.
Online Resource
This section is available to add information about an online resource for the metadata related to the collection, including web links (Fig. 25). If you have metadata related to the collection in another location, it can be linked here. The only required field is the URI (internet address, or URL), but other information such as Title, Description, etc., are highly recommended, if available.
Figure 25: mdEditor Online Resource section (empty).
More information about the Metadata tab can be found in the reference manual:
Keywords Tab
Keywords Tab
A major goal of ReSciColl is to make collections FAIR (Findable, Accessible, Interoperable, and Reusable). To promote the FAIR principles for collections in ReSciColl, the NGGDPP has connected several online controlled vocabularies (for example, those of the USGS Thesaurus) to the Keywords feature in mdEditor. These controlled vocabularies make resources more findable, accessible, interoperable, and reusable. We strongly encourage users to add as many relevant keywords as possible to their collection records using this feature.
To access this feature, select the Keywords tab in the mdEditor menu bar. You’ll be presented with the screen shown in Fig. 26. First, add a thesaurus by clicking either of the “+ Add Thesaurus” buttons.
Figure 26: mdEditor Keywords tab (empty).
What mdEditor defines as a thesaurus is actually a metadata object that may contain a selected keyword list. Several keyword lists are available by default in mdEditor, and more can be accessed through the different NGGDPP domain profiles.
Once you’ve added a thesaurus, an available keyword list can be selected from the drop-down list (Fig. 27). In the figure, the cursor is pointing at the “?” icon to the right of the “Collection Theme” vocabulary/keyword list. When available, these query symbols will bring up a pop-up displaying a brief definition of the term. Click on the keyword list name to the left of the “?” to assign a particular keyword list to the thesaurus.
Note: Only one keyword list can be assigned to a thesaurus object in mdEditor. If you wish to add keywords from more than one list, you will need to add a thesaurus for each list. If you select a particular list in error, you can change it via the drop-down menu up until you have added keywords from the list to the thesaurus. Once keywords have been selected from a list, if you no longer want that list in the metadata record, you will need to delete the thesaurus object.
Figure 27: mdEditor keyword list selection drop-down for Thesaurus object example.
After selecting a keyword list, you can add individual keywords to the metadata record by clicking the “Edit Keywords” button (Fig. 28).
Figure 28: mdEditor Thesaurus object with keyword list selected without keywords added.
The “Editing Thesaurus” screen will be presented, with the number of the thesaurus object, along with the name of the keyword list being used (Fig. 29; in this case, the USGS Thesaurus).
Figure 29: mdEditor keyword selection example.
As with the keyword lists, categories, subcategories, and terms may have definitions available by pointing to the “?” icon to the right of the entry.
If the keyword list is hierarchical, the categories and subcategories can be expanded using the “+” icons to the left of the category name. If the list is flat, or you have reached the lowest level, no “+” icon is shown (Fig. 30). An expanded section can be collapsed by clicking the “–” icon that appears when expanded. In Fig. 30, the “sciences” category has been expanded, along with the “earth sciences” subcategory, and the “geography” subcategory. The terms “cartography” and “paleogeography” are at the lowest level in this part of the hierarchy, representing individual keywords.
Figure 30: mdEditor hierarchical keywords list example.
To select a keyword to add to the collection metadata record, click the keyword itself. It will highlight, and it will appear, along with its hierarchy, if any, in the “Selected Keywords” section (Fig. 31).
Figure 31: mdEditor keywords list with keyword selected.
Continue adding keywords from the current list, if desired, either by navigating the list with the “+” and “-” icons and clicking on terms to add them, or by clicking on the "Search" tab above the displayed keywords or categories, typing in a search term, and then clicking on one or more of the desired results. Terms that represent a category or subcategory can be used as individual keywords and can be added to the "Selected Keywords" list by clicking on the term itself rather than the icons to expand or collapse the category. When finished with a particular keyword list and its associated thesaurus, click the “Back to List” button to the right of the “Selected Keywords” section (Fig. 32).
Figure 32: mdEditor keywords list with multiple keywords added.
mdEditor will return to the list of thesauri currently in the record (Fig. 33), from which you can add terms from a different keyword list by adding another thesaurus object. A second thesaurus object has been added in Fig. 33, ready for the user to select a desired keyword list. The "Custom Thesaurus" option may be selected if any desired keywords are unavailble.
Figure 33: A complete mdEditor Thesaurus object with multiple keywords, and a blank Thesaurus object ready to be filled out with words form a different list.
For more information about the Keywords tab, refer to the reference manual:
Extent Tab
Extent Tab
Extents are used in mdEditor to describe the geographic bounds of the resource (temporal extents can be defined via the Time Period object on the Main tab). Important: For ReSciColl collections, at least one geographic extent is required. It can be of any type or feature (e.g., bounding box, polygon, point), and an extent can contain more than one feature. Instructions for creating extents can be found in the mdEditor tutorial available here: https://guide.mdeditor.org/introduction/tutorial.html. An example is shown in Fig. 34.
Figure 34: mdEditor Extent object example.
Additional information about Extents can be found in the online reference manual:
Lineage Tab
Lineage Tab
Metadata in this tab are used to document the history of the resource, i.e., any steps or actions taken with the collection from acquisition through publication of the metadata. Fig. 35 shows the Lineage tab before a Lineage object has been added. To add lineage information, click either the "+Add Lineage" button or the "+Add Lineage Object" button.
Figure 35. mdEditor Lineage Tab main window (empty).
Fig. 36 shows an initial Lineage object with a brief statement about what was done, and in the Scope field, a value of "collection" has been selected.
Figure 36. mdEditor Lineage object.
If the collection example in Fig. 36 was updated at a later date, an additional Lineage object should be added with a statement describing the update.
Additional information is available in the mdEditor reference manual:
Distribution Tab
Distribution Tab
Metadata entered here should provide information about how to obtain or otherwise access the resource (e.g., website information, contacts).
From the Distribution Tab, a Distribution can be added by clicking either of the "+Add Distribution" buttons. As depicted in Fig. 37, a Distribution can have a distributor, a description, and/or a liability statement. Distributors are selected similar to other contact references in the metadata record, including selecting a role (usually "distributor" in this case) and a contact. A description is a text statement that is optional unless no contact is added for a distributor. A liability statement describes what is assumed or exempted by the collection owner.
Figure 37. mdEditor Distribution object with description, liability statement, and distributor contact.
Refer to the reference manual for more information:
Constraints Tab
Constraints Tab
Here, ReSciColl users should describe any constraints on the resource, including copyright requirements, intellectual property, use limitations, security concerns, etc. To add constraint information, click either of the "+Add Constraint" buttons. mdEditor will present the Editing Constraints window, as shown in Fig. 38.
Figure 38. mdEditor Constraint Editor screen.
Select from one of the available types of constraint in the drop-down list for "Constraint Type." Available constraints are:
- Use: Limitations of the fitness of the collection. I.e., what the data should NOT be used for. Use limitations are not legally binding and do not deal with security concerns.
- Legal: Legally binding constraints on the use and distribution of the collection, including copyrights and intellectual property rights.
- Security: Handling restrictions imposed on the collection for national security or similar security concerns.
More than one constraint type can be added to a collection metadata record. For legal constraints, mdEditor also provides a subsection that allows you to select from a drop-down list of access and use constraints, or to describe another type of constraint. You can also add a text description of use limitations, and one or more contacts for each constraint as a responsible party. Fig. 39 shows an example of a legal constraint with both unrestricted access and use, along with an administrator contact as a responsible party.
Figure 39. mdEditor Legal Constraint example.
More information is in the reference manual:
Associated Tab
Associated Tab
On this tab, users can enter information about any resources related to the main resource, including websites, publications that used data or materials associated with the collection, ScienceBase data releases, other collections, etc. Items within the main resource collection are not entered here. That process will be described in the "Item Metadata" sections of this protocol, Steps 40 - 50.
After adding an associated resource object by clicking the "+Add Associated Resource" button, mdEditor presents the screen show in Figs. 40 and 41. "Association Type" is required, and the associated resource's relationship to the collection can be selected from the drop-down list. "Initiative Type" is not required, but provides additional information about the type of resource being associated with the collection, along with the "Resource Type" section.
In "Basic Information," provide the title of the associated resource and any dates associated (Fig. 40.)
Figure 40. mdEditor Associated Resource editor, upper part.
Scrolling down reveals additional sections of the associated resource metadata, including "Responsible Parties," which can be selected from available Contact records, online information such as a web link for the associated resource ("Online Resource"; Fig. 41), any identifiers (e.g., DOI, IGSN) for the resource, series information (publication name, issue, and page), and any other details desired.
Figure 41. mdEditor Associated Resource editor, lower part.
For more information, consult the reference manual:
Documents Tab
Documents Tab
Here, users should provide metadata about any supplemental documents. These may include factsheets, data catalog pages, additional publications, websites, etc. Generally, documents listed here do not have a direct relationship with the collection (those would be added as "Associated Resource" objects), but may prove helpful in understanding and/or using the collection.
Fig. 42 shows the Additional Documents editing screen, which is similar to the Associated Resource editing screen, but with fewer elements. A Resource Type can be selected, a title is required, and Responsible Parties, Online Resources, and Identifiers can be added similarly.
Figure 42. mdEditor Documents editing screen.
Refer to the reference manual for more information:
Funding Tab
Funding Tab
On this tab, users should describe the funding for the resource. First, add a Funding Period, and within that, create an Allocation. In the Allocation box (Fig. 43), enter the Amount (in even dollars, with no decimal point or cents), the Award ID, and the Source. For Source, select from the contacts drop-down list. Important: ReSciColl users should import a contact for the NGGDPP for use here (refer to the Contacts section). A standard contact for NGGDPP is provided with the migration files for each organization.
Figure 43: mdEditor Funding Period Allocation dialog (empty).
Further information is available in the reference manual:
Settings
Settings
mdEditor has several settings that control its operation. They can be accessed from the “Settings” menu item in the upper right of the mdEditor window (Fig. 44).
Figure 44: mdEditor dashboard with Settings menu item highlighted by callout box.
Here, users can control aspects such as date format, start of fiscal year, and the mdEditor cache. Warning: Clearing the cache will delete all Metadata Records, Contacts, and Dictionaries from the mdEditor browser instance being used. Back up (Export) any desired records before clearing the cache. Fig. 45 shows the General Settings section of the Settings screen, including the button used to clear mdEditor's storage cache.
Figure 45: mdEditor General Settings options.
Additional information on mdEditor Settings is available in the reference manual:
Export
Export
mdEditor supports the ability to save the currently loaded Metadata Records, Contacts, and Dictionaries either as an mdEditor file or an mdJSON file. These files can then be shared with collaborators or saved to a local workstation for backup and/or archiving. Important: ReSciColl requires an exported mdJSON file of a collection to be submitted for publishing the record. More information about this can be found in the "Submit for ReSciColl Ingest" section of this document.
By default, exported files will be saved in your computer's "downloads" folder. This default location can be changed in your browser's settings, generally under the "Downloads" section. You can export all records or select from those currently available. You can also include the current set of mdEditor Settings in the exported file(s).
Refer to the video below for a walk through the mdEditor Export function (for a version of the video with supported closed captions and a transcript of the audio, see the USGS NGGDPP Data Submission page at this link: https://www.usgs.gov/programs/national-geological-and-geophysical-data-preservation-program/data-submission):
To export records, select “Export” on the upper menu bar of the mdEditor window.
Refer to the online reference manual for further information:
Import
Import
Complementary to exporting, mdEditor also supports importing records into a user’s local browser instance. Imported metadata must be formatted as either an mdEditor file (.json), mdJSON file (.json), or FGDC CSDGM file (.xml). Files can be imported locally (from the user’s PC or connected storage) or remotely over the web. The difference between an mdEditor and an mdJSON file is in what each contains. An mdJSON file is a single metadata record with associated contacts, while an mdEditor file is a JSON-formatted file containing multiple metadata records and/or contacts, whether or not the latter are associated with one of the included metadata records or not.
To import one of the accepted files, click the large blue button that takes up most of the Import Data screen. The button includes the text "Click or Drop a file here to import data" as well as the list of accepted formats immediately below (Fig. 46).
Figure 46: mdEditor Import Data screen with callout boxes highlighting features, including the large local file import button.
Next, you’ll be presented with the Import Data selection screen (Fig. 47), where you can choose which records and/or contacts to include, and whether to replace records currently loaded into mdEditor in the browser, or to merge the imported records with those currently loaded.
Note: "Replace" will replace any metadata records and/or contacts currently loaded. "Merge" will add new metadata records and/or contacts to those currently loaded. It will also update with the imported records and/or contacts any currently loaded records and/or contacts that have the same UUID.
Figure 47: mdEditor Import Data selection window.
More information about importing metadata in mdEditor is available in the reference manual:
Prepare for ReSciColl Ingest
Prepare for ReSciColl Ingest
When you are ready to submit either your new or existing, updated collection metadata record to ReSciColl, you will first need to save the record. In the title bar of the example in Fig. 48 is a red circle icon with an exclamation point. Pointing to this icon pops up up the message, “This record has been modified! Click to save.” You can then either click that icon, or the larger “Save” button on the upper right.
Figure 48: mdEditor Editing screen with need-to-save icon highlighted by blue circle.
If your collection metadata record contains errors (usually related to missing required elements), you will find an orange triangle icon with an exclamation point to the right of the collection title in the viewing and editing windows (Fig. 49).
Figure 49: Closeup of mdEditor Editing window with errors icon highlighted by blue circle.
To access the error messages, click on the error triangle. A screen like the one in Fig. 50 will appear.
Figure 50: mdEditor error messages list display.
The error message in Fig. 50 indicates that the collection metadata record is missing the “resourceType” element. Beneath the error message is the location in the hierarchy within the metadata schema. In this example, “metadata/resourceInfo” refers to information about the resource, which can usually be found on the Main tab in mdEditor. Paths listed beneath each error message on the mdEditor error screen will indicate the appropriate mdEditor tab containing the error. Click “Close” on the upper right of the error screen to return to the mdEditor main window. As depicted in Fig. 51, the Resource Type is missing from the example record.
Figure 51: mdEditor Editing window showing missing required element (Resource Type).
In Fig. 52, the error has been corrected, and the error triangle at the top no longer appears.
Figure 52: mdEditor Editing window showing previously missing required element (Resource Type) has been added, and the error icon no longer appears.
Once all errors have been corrected and the record has been saved, you will need to export the record so that you have an external copy, both to upload to ReSciColl, as well as for backup purposes or in case others wish to edit or need to approve the record on a different computer. Export and import functions are covered in previous sections and in the online tutorial:
For records destined for ReSciColl, on the Export Data screen of mdEditor, select only the metadata record for the collection you wish to submit to ReSciColl, without selecting other metadata records or contacts (Fig. 53). Contacts which have been associated with the desired metadata record will be included without the need to select them from the Contacts list.
Note: The "Include Settings?" button on the right, beneath the export buttons, should also be set to "No," as local browser settings for mdEditor are not relevant in ReSciColl.
Next, to export the file for ReSciColl publication, click the "Export mdJSON" button in the group of buttons on the upper right (Fig. 53).
Fig. 53. mdEditor Export Data screen with one metadata record selected.
Submit Collection Metadata to ReSciColl
Submit Collection Metadata to ReSciColl
Once you have exported your collection metadata record, you are ready to submit. Visit the ReSciColl homepage at https://webapps.usgs.gov/rescicoll/index.html, and in the lower right corner, near the bottom of the Collections Management section, under "Collection and Item Dashboard," click the "Use the Dashboard" button (Fig. 54, highlighted by yellow oval).
Figure 54. ReSciColl homepage with "Use the Dashboard" button highlighted by yellow oval.
Next, you will need to login with your ReSciColl account, which is your login.gov account (Fig. 55). Department of Interior users will select the "Log in with BisonConnect" option, while other users will select "Log in with login.gov." If you need assistance obtaining a login.gov account, contact login.gov via their website. Contact NGGDPP with your login.gov username once you have obtained an account, so it can be added to the ReSciColl user accounts. If you need further assistance logging into ReSciColl, contact NGGDPP.
Figure 55. ReSciColl login screen.
The ReSciColl dashboard (Fig. 56) will be presented. To publish a collection metadata record mdJSON file, click the "Upload mdJSON" button.
Figure 56. ReSciColl Collection and Item Management Dashboard
Select your organization from the drop-down list in Fig. 57.
Figure 57. ReSciColl Upload Organization selector screen.
Note: Your ReSciColl account must have permission to publish collections and items associated with your organization. If you have issues, please contact the NGGDPP for support and assistance (contact information is at the end of this document).
The screen will update to add a section for choosing the mdJSON file you wish to upload (Fig. 58). Click the "Choose File" button to bring up a file browser and navigate to the mdJSON file you exported from mdEditor. Once selected, the filename should be listed in the box to the right of the "Choose File" button. Click the "Upload" button at the bottom.
Figure 58. ReSciColl Upload screen.
If ReSciColl encountered any errors during upload, a message of "The file encountered errors during the upload process" will be displayed. If this is the result, you may wish to return to editing the file in mdEditor, checking for any errors that may have been missed and/or re-exporting it. Important: Remember not to export multiple collections at once or any Contact records or Settings along with a record intended for ReSciColl publication, and to select mdJSON as the export format.
If the upload is successful, a message indicating success will be shown, and the collection's page can be immediately viewed in ReSciColl under your organization's collections (Fig. 59).
Figure 59. ReSciColl Collection Page.
Item-Level Metadata Elements
Item-Level Metadata Elements
Once the collection is available on ReSciColl, such as the example shown in Fig. 59, you may add metadata for items within the collection, and you may also add attachments (e.g., images, documents).
The tables in this section list the nineteen (19) metadata elements accepted by ReSciColl. Some metadata elements are mandatory to adequately describe records. Other metadata elements are optional but recommended, when available, to enhance metadata records for ReSciColl users.
| A | B | C | |
| Element Name | Definition | | |
| localID | Item identifier provided by organization (e.g., Oil and gas well API, Well no., Local ID, Map ID, Publication No., Sample ID). | 1 | |
| title | The name/title of the item in the Collection (e.g., map name, sample identifier). | 1 | |
| abstract | Main description element. May be anything from a few words to an essay, suitable for a general audience (i.e., who, what, where, when, and why of the item in the collection). | 1 | |
| datatype | A controlled vocabulary of data types (keywords; e.g., Auger Samples, Fluid Samples, Geochemical Samples, Hand Samples, Ice Cores, Paleontological Samples, Rock Cores, Rock Cuttings, Sediment Cores, Sidewall Cores, Thin Sections and Polished Sections, Type Stratigraphic Sections). Use the NGGDPP Collection Category terms here: https://www.sciencebase.gov/vocab/categories?parentId=4f4e475ee4b07f02db47df22. An item may include multiple dataTypes, separated by commas (no spaces), and grouped between double quotes if using CSV format (refer to Step 41 below for Excel format; e.g., "Auger Samples,Rock Cuttings,Sidewall Cores"). | ≥1 | |
| coordinateLon_WGS84 | Geographic longitude in decimal degrees, WGS84. Example: -118.023423. | 1 | |
| coordinateLat_WGS84 | Geographic latitude in decimal degrees, WGS84. Example: 45.02312. | 1 | |
| publicationDate | Publication date regarding the currency of the metadata, which may be when the item metadata are added/updated to the collection record in ReSciColl. May be of any of the following formats: YYYY, YYYYMM, or YYYYMMDD. Ranges are not valid for this field. | 1 |
Table 1: Required metadata
| A | B | C | |
| | Definition | | |
| alternateTitle | Additional title information and/or identifiers for individual items in a collection may be provided for further identification or use by other web service interfaces. The alternateTitle field may include either textual titles or specific sample IDs used by the collection. | 1 | |
| alternateGeometry | The underlying item resource may use an alternate method of storing a geospatial footprint. If so, this text field should be used to describe the authoritative source for geographic location and how the simple coordinates were derived. Examples of authoritative sources include: Getty place names – Linked open data (LOD) resource, includes API and SPARQL queries (http://vocab.getty.edu/queries); USGS GNIS: U.S. and Territories only, search through U.S. Board on Geographic Names (https://geonames.usgs.gov/apex/f?p=138:1::::::); USGS GAZ – Connected to GNIS, U.S. names (https://edits.nationalmap.gov/apps/gaz-domestic/public); Library of Congress terms – A linked data service (https://id.loc.gov/vocabulary/identifiers/geogndb.html) | 1 | |
| onlineResource | URI (Uniform Resource Identifier or web link) of an item-level data source. | ≥1 | |
| browseGraphic | One or more URI pointers to images representing the specific record. | ≥1 | |
| date | If a meaningful date within the geosciences domain can be attached to the record (e.g., collection date, map publication date), it can be supplied here. Date may be to any degree of precision or may be left blank to indicate uncertainty. Examples: 2001, 2001-2003, 1939-1945, 20030331, 20000331-20000401. | ≥1 | |
| verticalExtent | Vertical extent is useful for describing rock core samples and borehole depths. Specification of extent must contain three elements, in the following order, with no spaces between them: unit of measure (meter, feet), maximum depth, minimum depth. Example: m,35.4,0 for a rock core measured at maximum depth of 35.4 meters from the surface. | 1 | |
| IGSN | International Generic Sample Number (IGSN) of the physical item in the collection (e.g., cutting, core, specimen). Only the IGSN itself is required, not the full web link (e.g., 10273/ICDP5054ESYI201, not http://doi.org/10273/ICDP5054ESYI201). | 1 | |
| parentIGSN | The IGSN of the parent material (sample) the derivative material (subsample) was obtained from. Only the IGSN itself is required, not the full web link. | 1 | |
| relIGSN | The IGSN of a related sample or subsample (e.g., duplicate, or split). Only the IGSN itself is required, not the full web link. | ≥1 | |
| relationType | Relationship of the IGSN to the largerWorkCitation. Use the DataCite Relation Type vocabulary terms here: https://www.sciencebase.gov/vocab/vocabulary/611d428bbfff3461918aba6e | ≥1 | |
| largerWorkCitation | Physical sample connection to publication (DOI). The collection may document a larger set of samples with only a section used in a journal article. Only the DOI itself is required, not the full web link. | 1 | |
| attachmentFileName | Full name of file that is to be linked to the item via the file attachment process. A full folder/directory path is not required, just the filename and extension. Must exactly match the name used during file attachment ingest. | 1 |
Table 2: Optional metadata
Map the sample level properties found in your collection to ReSciColl metadata elements by creating a Microsoft Excel or CSV file with column headings that match the metadata elements described in Tables 1 and 2 with rows containing the metadata for each item in the collection. An Excel template is available to download here. In some cases, this might be a one-to-one mapping. For instance, the “date” element might correspond to the 'DateCreated' property in your collection database. For the abstract, it may be desirable to concatenate several fields to provide a richer description of the resource and more specific search results. If the location information is stored in township/range/section, it will be necessary to convert these values to a single geographic latitude and longitude point to populate the required WGS84 coordinates metadata elements. The descriptive township/range/section may then be provided in the alternateGeometry field.
Item-Level Metadata Upload Format
Item-Level Metadata Upload Format
ReSciColl Dashboard supports two primary formats for loading data into ReSciColl. Data providers may submit Comma Separated Value (CSV) formatted files or XLSX (Microsoft Excel) files. These formats are discussed in more detail below.
File Format
A CSV file format represents tabular data. Each row in the file corresponds to a single metadata record, and the properties associated with each record are separated by a comma. The comma is a special character that separates data fields, which is useful for recording item/sample characteristics. However, comma delimited files present issues when data fields include commas, which requires enclosing entire fields in double quotes (“) so that the embedded commas are not regarded as field delimiters but as part of the field information. For properties that contain more than one value, the ReSciColl Item Ingest Tool requires data providers to use double quotes around fields that contain values within a single element. For example, a single record with five fields might look like the following, using double quotes as special delimiter characters:
1341234,This is a Title,”m,35.4,0”,”Please note, the existence of a comma within.”,”2020-10-01,2021-04-02”
The above line contains five fields, three with multiple values enclosed in double quotes, with the final field containing two dates.
CSV formatted files must also include a header on the first line that indicates the metadata element names and delimiters corresponding to the subsequent data (similar to a map legend). Applying this to the above example, the first two lines of the CSV file would look like the following:
localID,title,verticalExtent,abstract,date
1341234,This is a Title,”m,35.4,0”,”Please note, the existence of a comma within.”,”2020-10-01,2021-04-02”
Most databases may output CSV formatted files through built-in utilities or customized queries. Microsoft Excel may be used to produce CSV formatted files, but the options for character encoding and delimiters are not as flexible as with other CSV file creation tools and must be considered carefully. Excel uses the system settings to determine character encoding and delimiters and will automatically enclose in double quotes any fields containing commas within, and for CSV files, Excel will automatically insert the commas between fields. On a Windows platform, delimiters may be set by accessing the Control Panel --> Regional and Language Options. This option is not available on a Macintosh. Excel file format (*.xlsx) may also be used, but observe caution with date formats, as dates of the text itself, not the field formatting, must follow the structure given in the tables in Step 40, e.g., YYYY, YYYYMMDD.
Note: If uploading a file in Excel (*.xlsx) format, make sure all the columns/fields are of either text, general, or number format. Do not use date or other specific formats, as that affects how the data are saved, and could cause validation errors in ReSciColl.
CSV file example is available here:
Item-Level Metadata Upload to ReSciColl
Item-Level Metadata Upload to ReSciColl
Create an Excel file (*.xlsx ONLY) or regular (not pipe-delimited) comma-separated value (CSV) file using the requirements described in the previous section. Note: When using Excel for either file format, the row number indicated by Excel is 2 rows off from the rows as numbered by ReSciColl. ReSciColl does not count the header row (with column names), so the first row of metadata (row 2 in Excel) is considered row 0 in ReSciColl.
After creating/updating your collection, navigate to the ReSciColl home page. The collection itself must be created/updated first to then hold all the items in the collection.
Click the “Use the Dashboard” button under “Collection and Item Dashboard” (Fig.60).
Figure 60. ReSciColl homepage with "Use the Dashboard button highlighted by yellow oval.
Log in to ReSciColl with your username and password (Fig. 61). USGS and other Department of Interior users should click the “Log in with BisonConnect” button, while external users should select “Log in with login.gov.” If you need assistance obtaining a login.gov account, contact login.gov via their website. Contact NGGDPP with your login.gov username once you have obtained an account. If you need further assistance logging into ReSciColl, contact NGGDPP.
Figure 61: Login screen for ReSciColl.
On the “Collection and Item Management” dashboard window that loads next (Fig. 62), click “Upload New Items.”
Figure 62. ReSciColl Collection and Item Management Dashboard.
Under “Choose a Collection,” from the drop-down list (Fig. 63), select the collection to which you want to add items, and when you have chosen the correct collection, select “Yes” in the “Is this the correct collection?” section.
Figure 63: ReSciColl Item Management collection selector drop-down menu.
Under “Validate Your CSV,” select “Choose File,” browse for the item metadata file you wish to upload, verify it is the correct file, and click “Validate” (Fig. 64).
Figure 64: ReSciColl Item Management collection verification dialog and item metadata file selector.
ReSciColl will validate the uploaded metadata file. If there are errors, you can fix them and try again by clicking the “Errors Fixed” button that will appear. If you have issues understanding and/or correcting the errors, contact NGGDPP for assistance. Once the file is correctly validated, a success message will be displayed (Fig. 65).
Figure 65: ReSciColl item metadata validation success message.
Notes:
- When uploading item metadata to a ReSciColl collection, ReSciColl creates a web page for each item (Fig. 66).
- If you have already uploaded items to the selected collection, ReSciColl uses the “localID” field in the item metadata CSV or XLSX file to determine whether or not an item already exists. If it finds a localID that matches, it updates that item. If it does not find a match, it adds the item to the collection.
- If you need to delete items from a ReSciColl collection, contact the NGGDPP for assistance.
Figure 66: ReSciColl Item webpage example, showing a map inset and links to a related publication, the parent collection page, and the organization page.
Upload File Attachments to ReSciColl
Upload File Attachments to ReSciColl
If you listed attached files in the item metadata template, you may now upload those via the attachment dialog box (Fig. 67). ReSciColl only accepts attachment uploads as a single file of 299 MB or smaller. For multiple attachments, a Zip file is required. If the total size of a Zip file of all your attachments is larger than 299 MB, you will need to upload the zipped files one at a time. ReSciColl will prompt you for each file. If your attachments are too large for these limits, please contact NGGDPP for assistance and possible alternatives.
Figure 67: ReSciColl Item Management screen with file attachment selector.
Repeat the process for each separate collection.