Import Your Data

With Conserv Cloud, you can import data from your existing data loggers.

Import Process

Conserv designed the import process to be hassle-free, but sometimes issues may arise. Understanding the overall process is key to identifying any problems.

The Conserv import process consists of the following steps:

1. Preparing your files for import.
2. Choosing a sensor type.
3. Selecting file(s) to import.
4. Mapping your data fields to Conserv Cloud.
5. Importing the data.
6. Mapping your import to new or existing sensors.

This article guides you through each of these steps.

 

Prepare Files for Import

If you use PEM2 or HOBO data loggers, read the following articles to get your files set up for import.

📝 Import Data from PEM2 and eClimateNotebook

📝 Import Data from HOBO UX & HOBOware

📝 Import Data from HOBO MX Bluetooth and HOBOlink

Time Zones and Timestamps

Data is presented to you in your specified time zone within Conserv Cloud. In some cases, files that are exported by data loggers may include a time zone that is specified in the header, but it lacks timestamps in the date/time fields. When this occurs, the import process defaults to Universal Time Coordinated (UTC), which can align with London time depending on the time of year due to British Daylight Savings. This discrepancy can lead to analytics issues because the time data may appear off by several hours.

The following timestamp-related problems may arise during import.

Problem 1: Incorrect Date Format

If your file has a date format like 03/19/23 05:47, the software may struggle to recognize it correctly and can interpret it as UTC time. This misinterpretation can result in timestamps showing several hours behind the actual reading time. To resolve this, change the date format for those cells to the following example:

• Date Time (in Google Sheets) or Custom yyyy-mm-dd h:mm (in Excel)
  so that your timestamp looks like this instead:
  03/19/2023 05:47:00 or 2023-03-19-11T05:47:00Z or 2023-03-19-11 05:47:00.

Problem 2: Missing Time Zone Offset

If your timestamp is formatted as "2023-03-19T05:47:00Z," the import process interprets it as UTC time. This means that your analytics may show readings several hours off, depending on daylight saving adjustments. The software assumes the reading is for March 19, 2023, at 5:47 UTC (approximately London time). To ensure that the data is accurately imported for your local time zone, the date/time value should include the time zone offset.

Types of Timestamps

The format of the timestamp in your .csv file varies depending on the software and brand of your imported data logger. Some brands include the time zone in the timestamp, while others do not. To prevent confusion related to time zone differences, Conserv strongly recommends that you create a separate column in your file with a UTC-corrected timestamp. This way, you can accurately map your data during the Conserv import process, ensuring that your analytics display values at the expected times.

Time Zone Conversion

To assist you with this conversion, Conserv provides a sample Google Sheets file that includes a formula for converting date/time values without a time zone offset into UTC. Additionally, the file features a second tab with examples demonstrating how the formula varies based on your specific time zone.

In the formula below, the "+ (H/24)" component indicates that the time in your original file is from a time zone that is H hours behind UTC. In other words, UTC is H hours ahead of you. For example, if it is 5:47 AM in your local time in EST, it is 10:47 AM in UTC.

If that formula returns an error, ensure that you solved Problem 1 first because the formula will not work if the original date format is not correct. If you still cannot solve the problem, make your own copy of the sample Google Sheets file, and try pasting your data there.

Clean Up Your Data Logger File

1. Avoid empty columns.
2. Remove all of the blank rows in the data. Review your data for missing values or errors.
3. Use consistent units of measurement.
4. Include only the data that you need.
5. Use the right data types, for example, temperature should be a number.
6. Use UTF-8 encoding for files.
7. Avoid special characters in header names, for example,. no emojis.
8. Different countries may use varying notations for decimals, which can confuse the software and lead to errors regarding invalid numeric reading data. If your file uses commas for decimals (for example, 38,75 instead of 38.75), change all of the commas to points before you import your files.
9. Be aware that Conserv SCS devices are set to take readings every 15 minutes. We do not recommend uploading data logger files with intervals lower than 5 minutes as data-heavy files can cause problems. Avoid uploading single files with more than 15,000-20,000 rows of readings.

Video Tutorial

You can watch this video to help explain preparing your files.

Select a Sensor Type

Conserv designed the import process to work well with any .csv files, assuming that you prepared the files.

1. Sign in to Conserv Cloud.
2. In the left navigation menu, select Import.
3. Click + (Import Data).
4. Select your sensor type.
   • If you use data from HOBO sensors, select Onset.
   • If you use data from PEM2s or eClimateNotebook, select IPI.
   • For all other .csv files, select Other Loggers.

5. Click NEXT STEP.

Select Files

If you use the free version of Conserv Cloud, the import is limited to one file at a time.

If you are a Conserv customer with a paid subscription, you can use the multi-file import to import up to 20 files at a time. If you have more files than that, you must perform multiple imports.

1. Drag your files to the dialog, or select the files to upload from your computer.
2. Click NEXT STEP. 

Map Data Fields

In the mapping dialog, the fields on the right represent the data that you are importing, while the fields on the left indicate where this data will be placed in Conserv Cloud. The software automatically attempts to map your fields to the appropriate locations in Conserv Cloud. Sometimes, this automation can make mistakes. Double check your mapping and manually adjust any mistakes by dragging and dropping the fields.

1. Drag the fields from Conserv (left) to match the fields from your file (right).
2. Click NEXT STEP.
   The import process begins.

The following video shows the drag-and-drop feature.

Sometimes the mappings are incorrect due to unusual file structures or languages. The simplest way to ensure a successful import is to transfer your data into the template that is provided by Conserv.

Import the Data

Conserv has a working limit of about 30,000 rows for data import files, making it sufficient for most use cases. If you need to import more data, consider splitting the file into multiple parts. We recommend files of no more than 20,000 rows at a time to ensure a smooth import process.

Additionally, disable pop-up blockers or whitelist the Conserv.io domain because some ad blockers or pop-up blockers may interfere with the import process. For instructions on how to whitelist Conserv.io, consult your browser or security product’s documentation.

Map an Import to New or Existing Sensors

1. To complete the import process, associate your imported data with a new or existing sensor in Conserv Cloud.

2. Click FINISH.

Support

If you have import issues, please create a support ticket.