Upload CSV Data

Here we describe the process for uploading inspection data via CSV into the Raptor Maps system. In order to properly tie the data to your account, we require a signed URL for each upload. We have provided API to enable this as an integration or one-time upload.

Process

The basic process is to collate data into a proper CSV, use an API to retrieve an access token, then use the token to upload the CSV to Raptor Maps' system.

Creating the CSV

Save your data in a csv format with the following headers. Fields marked as required need to have a value supplied in the data. All headers must be present, even if the data in that column is optional, and columns must be in the same order as presented in the table.

Header	Data Type	Required
farm_id	string	true
farm_name	string	true
timezone	string	false
commissioning date	YYYY-MM-DD HH:MM:SS	false
farm_latitude	float	false
farm_longitude	float	false
equipment_id	string	true
equipment_name	string	true
equipment_type	enum - see table below	false
sensor_id	string	false
sensor_name	string	false
sensor_type	enum - see table below	true
observation_time	YYYY-MM-DD HH:MM:SS	true
measured_value	float	true

Equipment Types

Sensor Type By Equipment

Equipment	Sensor Type	Units	Interval (minutes)	Aggregation (Avg,Sum, etc)
Inverter
	dc_voltage	V	15	Average
	dc_current_amp	A	15	Average
	ac_power	W	15	Average
	dc_power	W	15	Average
Combiner
	dc_voltage	V	15	Average
	dc_current_amp	A	15	Average
	ac_power	W	15	Average
	dc_power	W	15	Average
Meter
	production_meter_energy	Wh	15	Maximum
	production_meter_power	W	15	Average
Weather Station/Pyranometer
	irradiance_poa	W/m^2	15	Average
	irradiance_ghi	W/m^2	15	Average

Requesting an Signed URL

After retrieving an API access token, use that token to make a request to the org integration endpoint documented below. The response will contain one or more objects that have an ID field.

  {
    "id": int,
    "org_id": int,
    "api_type": integration_name,
    "auth_info": {
      "username": user_name,
      "password": password
    },
    "query_interval_secs": integer,
    "settings": {},
    "enabled": boolean
  }

Use the ID to send a request to the GET Signed URL for Data Upload endpoint. That signed URL is the target of a PUT request we use to transport the CSV to Raptor Maps. The CSV in question can be no larger than 5GB. If it is larger than this it must be split up into multiple PUTs which will require multiple signed url requests.

Python Example

import requests
import os
import sys
FILE_TO_UPLOAD = sys.argv[1]
INTEGRATION_ID = <int>
# Authenticate with an API token provided to you by Raptor Maps
token = os.environ['RM_API_TOKEN']
# Specify API endpoint - RM Add Integration workflow will also provide this!
endpoint = f'https://api.raptormaps.com/insights/api/v1/integrations/org_integrations/{INTEGRATION_ID}/signed_url'
# Specify request headers
headers = {
  'content-type': 'application/json',
  'Authorization': f"Bearer {token}"
}
# Make HTTP GET request to get Upload URL
resp = requests.get(endpoint, headers=headers)
# Parse response
url = resp.json()
print(url)
# Upload file
headers = {
  'content-type': 'text/csv'
}
with open(FILE_TO_UPLOAD, 'rb') as f:
  resp = requests.put(url, data=f, headers=headers)
print('Upload Complete!')