Best Practices for Managing Your Data
Following best practices for managing your research data can ensure it will be available to other researchers in the long term. The following best practices are suggested guidelines, and some recommendations will not apply to every discipline or project. Overall, however, following these guidelines will streamline your data management activities, and help prevent data loss.
Folder and file organization
- Choose a consistent organizational structure for all your project folders
- Structure your directory hierarchies logically (e.g., Project/Site/Year/)
- Your directory structure shouldn’t alone provide the necessary context for your file contents (e.g., Project/Site/Year/data.txt is less useful than, Project/Site/Year/2015_data.txt).
File naming conventions
- Determine your file naming convention before your data gathering begins
- Give your files descriptive file names that describe important aspects of your project (such as Project, Content, Location, Date, Time, or Version Number)
- Be consistent across all project files and folders
- Use underscores rather than spaces
- Never use special characters such as “ / \ : * ^ & $ #
- Use a consistent date convention such as YYYYMMDD, and a consistent time convention such as hhmmssTZD
- Use non-proprietary, uncompressed file formats for archiving
- Use .txt instead of .doc
- Use .csv instead of .xsl
- Use .tif instead of .jpg
- Use .pdf instead of .ppt
Metadata and Project Documentation
- Define a data dictionary for your project that will help you maintain consistency in how you describe and categorize your data
- Use a metadata standard that corresponds with your discipline
- Create a README file for your project that details the project specifics (e.g., the PI for the project, what the project is about, when the project was undertaken, and the rights associated with the project—such as a Creative Commons license)
- Keep multiple backups of all your data and project documentation (e.g., on local computer, on external hard drive or disc, on network storage)
- Other options include Box.com, access to which is provided by UC Merced IT, as well as the SDSC Cloud Storage, which provides affordable, competitively priced options for UC researchers
Provisions for Confidentiality/Restricted Data
- Evaluate the sensitivity of your data: Researchers should consider whether or not their data contains either direct or indirect identifiers that could be utilized with other public information to identify research participants.
- Comply with UC regulations: UC researchers concerned about confidentiality issues with their data should consult the appropriate UC Requirements and Guidance for Conducting Research Involving Human or Animal Subjects.
- Comply with regulations for health research: HIPAA Privacy Rule, Information for Researchers.
- Protect the privacy of student data: FERPA (Family Educational Rights and Privacy Act).
- Enable restricted use of your data: Do you want to make your data available in a more restricted, limited-access manner? The ICPSR DSDR program has resources for data producers.
- Learn about guidelines from the National Academy of Engineering: Their Online Ethics Center includes a discussion of Ethical Issues in Data Management.
Best Practices from DataONE
Offers a Best Practices Primer as well as a searchable database of recommendations for managing data through all stages of the data lifecycle.
IMBER (Integrated Marine Biogeochemistry and Ecosystem Research) Data Management Cookbook
Best Practices for Preparing Environmental Data Sets to Share and Archive from Oak Ridge National Laboratory Distributed Active Archive Center, Oak Ridge, Tennessee, U.S.A. doi:10.3334/ORNLDAAC/BestPractices-2010
Guides to Good Practice by UK Archaeological Data Service and Data Antiquity