Integrations & Data Collection

This document provides a guide to how the platform integrates with external systems to collect and process data. It covers both temperature sensor providers and simulation-based integrations like COMSOL, which work together to power Coldtivate’s core features around monitoring and shelf-life prediction.

1. Temperature Sensor Integrations

These integrations allow the platform to collect accurate, real-time temperature data from cold storage units.

Viewing temperature and humidity history

Operators, Registered Employees, Site Officials, and cooling users can use the room conditions graph to review environmental readings for a cooling unit. Cooling users only see readings for the cooling units they are linked to; for any other unit the graph shows no data.

Open Cooling Units and select Room Conditions.
Select the cooling unit you want to review. Changing the cooling unit resets the graph to Temperature and the 24H range.
Select Temperature or Humidity above the graph. Temperature values are shown in degrees Celsius (°C), while relative humidity values are shown as a percentage (%). If the selected cooling unit has humidity readings but no temperature readings, the view automatically opens the humidity graph.
Press and move across the graph to inspect a reading. The value card above the graph shows the selected value, date, and time. When you release the graph, the card returns to the latest reading in the selected period.

The graph initially shows the latest value in the selected period. A line connects multiple readings; a period containing only one reading is shown as a single point. Pull down on the page to request the latest available data.

Selecting a time range

Use the tabs below the graph to control the period displayed:

Range	Usage
24H	The default view. Shows readings from the preceding 24 hours.
7D	Shows readings from the preceding seven days.
CUSTOM	Shows readings between a start and end date that you select. Both dates are required before data is loaded.

Custom ranges can cover any period of up to seven days since the cooling unit was created:

The start and end dates cannot be in the future.
The start date cannot be earlier than the date the cooling unit was created.
The selected period cannot exceed seven days. If it does, the app keeps the previous end date and asks you to choose a shorter period.
Changing the start date clears an existing end date when the resulting range would be invalid. Select a new end date to continue.

While data is being requested, wait for the loading state to finish before changing the metric or period again. If no readings exist for the selected metric and period, the graph displays No data available and identifies whether the empty period is the last 24 hours, the last seven days, or the custom period. Select another period to look for historical readings. A newly connected sensor may also need time to transmit its first reading.

Usage examples

Review a recent temperature change: Keep Temperature and 24H selected, then move across the graph to compare individual readings and their timestamps.
Check humidity conditions: Select Humidity. Confirm that the value is shown as a percentage, then review the graph for sustained high or low readings.
Investigate a known event: Select CUSTOM, choose the event date as the start date and an end date up to seven days later, then inspect readings around the event time.

Responding to stale sensor data

For a cooling unit with a connected sensor, the page shows when the temperature data was last updated. The data is considered stale when the backend reports a sensor error, when there is no temperature reading yet, or when the latest temperature reading is more than 12 hours old.

When data is stale:

Last updated appears in red.
A red Stale badge with a clock warning icon appears below it.
Selecting the badge expands a Sensor data stale warning. When the last reading time is available, the warning reports approximately how many hours or days have passed since the update.

The warning text depends on your role:

Registered Employees are directed to Management → Cooling Units to check the room's sensor connection.
Everyone else (Operators, Site Officials and cooling users) is asked to coordinate with a Registered Employee to check the sensor connection.

Do not treat the displayed value as a current room reading while the warning is present. Note the cooling unit and its Last updated time, and follow the instruction in the warning. Once the connection is restored, pull down to refresh the page. Confirm that the timestamp advances and the Stale badge disappears before relying on the reading as current.

Frontend Integration

Form Management:

Sensors are configured through forms built with react-hook-form, ensuring robust state management, validation, and user-friendly input handling. All providers ask for the user's account credentials. Figorr, Ubibot and Victron share one generic form, and their credentials are stored to poll the provider.

Ecozen and Inficold each have their own form that runs the whole setup inside the sensor sheet: a provider sign-in that only discovers the account's devices, device selection with Test connection, then saving. The sign-in credentials are used only for discovery and are never stored (see Credential Storage).
API Connectivity:

The frontend is built with React (and React Native) and is responsible for collecting user input and passing it to the backend through a centralized SensorsService.

Sensor Setup Flow:

The user selects a sensor provider (Ecozen, Figorr, Ubibot, Victron, or Inficold).
Figorr, Ubibot and Victron: the user fills in credentials (email/username and password). The frontend calls the unified sources endpoint, which validates the credentials and returns every sensor on the account. The user selects one.
Ecozen:
- The user signs in with their Ecozen Cloud username and password. The app sends them only to the sources endpoint, and clears them once the request finishes.
- The app lists the account's machines by label and ID in a searchable list. There's no machine-ID field and no chamber selection.
- The user selects one and presses Test connection. This checks the machine through Coldtivate's own OAuth client and saves nothing.
- When the test succeeds, the user saves. The saved sensor data holds the machine ID and the short-lived source ticket returned by the test, never the Cloud credentials.
- Each failure has its own message. Wrong credentials, no machines, an unsupported sign-in step, discovery unavailable, a machine whose API access isn't enabled (directing the user to Coldtivate support), no recent readings, and an expired session. See the Ecozen Integration Guide.
Inficold:
- The user signs in with their Inficold dashboard username and password. The app sends them only to the sources endpoint, and clears them once the list is shown or the form closes.
- The app lists the account's devices by name and ID in a searchable list. The user selects one and presses Test connection, which checks the device against Inficold without saving.
- When the test succeeds, the app lists that device's chambers. The user selects one and saves.
- The saved sensor data holds the chamber's source ID and the short-lived source ticket returned with the chambers, never the dashboard credentials.
- Each failure has its own message: wrong credentials, no devices on the account, a selected device that isn't enabled for Coldtivate (named, with its ID), Inficold unavailable, and an expired session. See the Inficold Integration Guide.
The selected sensor is saved with the cooling unit when it is created or updated.

In summary, the frontend is responsible solely for gathering and transmitting sensor-specific data, while the backend takes care of establishing sensor connections and handling all subsequent operations. More on that, in the next section.

Backend Integration

The backend handles authentication, sensor listing, periodic temperature syncing, and error handling. Key aspects of the architecture include:

Unified Sensor Listing API:
A single endpoint (storage/v1/user-sensor/sources/) receives the sensor type (integration_type) and credentials (username, password).
For Figorr, Ubibot and Victron, the backend authenticates with the provider and returns the account's sensors (id and name).
Ecozen and Inficold use the same endpoint and payload shape, with the user's portal credentials. The backend signs in to the provider's portal only to list the account's devices, and makes no data-API calls. The response adds a signed device_token (device ticket) for the provider's connection test.
Provider connection tests:
Ecozen: storage/v1/ecozen/test-connection/ with {"source_id": "<machine_id>", "device_token": "<device ticket>"}. It checks the machine through Coldtivate's OAuth client and the v3 API. It returns a signed source_token (source ticket) that's required to save. See the Ecozen API endpoints.
Inficold: storage/v1/inficold/test-connection/ with {"device_id": 12345, "device_token": "<device ticket>"}. It checks the device against the official API and returns its chambers with a source_token. See the Inficold API endpoints.
Both save nothing, return {code, detail} errors with stable codes, and use 409 rather than 422 for a device that isn't enabled.
Tickets last 15 minutes and are bound to the user and to the devices or sources they list. When saving the cooling unit, both providers check the source ticket and the user's staff role in the cooling unit's company. Inficold also rechecks the device live; Ecozen trusts the recent successful-test ticket and makes no provider request during save. See Ecozen setup tickets and Inficold discovery tickets.
Scheduled Data Syncing: the sync_sensor_data dispatcher queues one poll_sensor_integration task per linked integration every 4 hours. See Polling and ingestion.
Error Handling & Notifications:
Transient failures (network errors, timeouts, rate limiting, 5xx and malformed responses) are retried after 5, 15 and 30 minutes. A provider's cooldown (for Ecozen, its Retry-After) is never cut short: polls wait towards it in steps, and polls stopped before sending anything are rescheduled without spending retries (see Ecozen rate limits). If all retries fail, the integration is marked as failed, a SENSOR_ERROR notification is created for the cooling unit, and the error is reported to Sentry.
Authentication, configuration and invalid-source errors are not retried. They include a rejected server token or OAuth client, missing server settings, and a device the provider won't give Coldtivate access to. They mark the integration as failed immediately, and are reported to Sentry only when a healthy integration first fails.
A successful poll clears that integration's error state. The cooling unit's SENSOR_ERROR notification is cleared once all of its integrations have recovered.
A result fetched for a sensor that was deleted or relinked during the request is discarded.
For Ecozen and Inficold, logs and Sentry reports carry static messages, error types and identifiers only, never credentials, tokens, request bodies or provider responses.
Credential Storage:
Figorr, Ubibot and Victron credentials are stored on the SensorIntegration record, with the password encrypted.
Ecozen and Inficold store an empty password. Their server credentials are backend settings, added at runtime (see Shared adapter contract). Portal credentials and setup tickets are never stored.
Sensor Deletion:
A DELETE endpoint allows removal of specific sensors if needed, with permission validation based on the user’s organization.

Polling and ingestion

Schedule: the Celery beat task sync_sensor_data runs every 4 hours, at minute 0 of every fourth hour (see Scheduled Tasks). It queues one poll_sensor_integration task for every sensor integration linked to a cooling unit that isn't deleted and has sensors enabled. Several integrations on one cooling unit are polled independently.
Poll window:
- Each integration keeps its own watermark (last_datum_at, last_successful_poll_at).
- A poll requests everything since the last stored reading, minus an overlap: 15 minutes, or 3 hours for Ubibot.
- Readings between polls are therefore not lost, as long as the provider still has them.
First sync:
- Figorr, Ubibot and Victron: pull the provider's full history when a sensor is linked, synchronously, during the save.
- Inficold: pulls the last 24 hours, because its API filters only by date and would otherwise page through every row.
- Ecozen: also covers the last 24 hours. It isn't run during the save: a poll_sensor_integration task is queued once the save's database transaction commits, so it uses the same credentials, retries and notifications as scheduled polling.
Raw readings:
- Each reading is stored in SensorReading with the provider's original timestamp in UTC.
- It's keyed by cooling unit, integration (source_key integration:<id>), timestamp and type, so repeated polls update a reading rather than duplicate it.
Normalization:
- Units: temperatures are converted to °C (Fahrenheit is converted). Relative humidity must be 0–100 %.
- Rejected values: non-numeric, non-finite, out-of-range and sentinel values (such as 999) are rejected and logged. When shared ingestion rejects a reading that has a usable timestamp, it deletes the stored raw reading for that integration, timestamp and type. Values an adapter discards before ingestion, such as Ecozen's missing or non-finite values, don't trigger that cleanup. Missing readings are skipped, never stored as zero.
- Set points: a set point, when the provider reports one, is normalized like a temperature.
Aggregation:
- Readings are grouped into 15-minute UTC buckets in CoolingUnitSpecifications, which the app graphs and features such as shelf-life prediction use.
- Within a bucket, each integration's readings are averaged first, then the integrations are averaged together, so each integration counts equally.
- The bucket's set point is the latest one reported in it.
- Buckets are rebuilt whenever their raw readings change, so there's no minimum spacing between stored readings.
Identity changes: changing a sensor's type or source ID resets its watermark, so the new source starts fresh. Credential-only changes keep it.

Sensor-Specific Behaviors

Below is a summary of each supported sensor type:

Ecozen

See the Ecozen Integration Guide for setup, configuration and troubleshooting.

Discovery: machines are found through the user's Ecozen Cloud account. Only the machine the user selects is then checked through the v3 API, with Coldtivate's OAuth client. Seeing a machine in the portal doesn't mean Coldtivate's client can read it.
OAuth only: all data requests use Coldtivate's client. There's no fallback to customer credentials or to the portal.
Readings: room temperature (Room_1_T), relative humidity (Room_1_H) and the set point (Set_T), attached to temperature readings with the same timestamp.
Timestamps: Unix milliseconds, stored in UTC. Requests are padded by 14 hours and readings are filtered to the exact window, because how Ecozen interprets request times hasn't been confirmed.
Errors: a machine that isn't enabled for Coldtivate's client is a non-retryable access rejection. A recognized empty-history response is a successful poll with no readings.

Figorr

Unified Endpoint Integration
Authenticates with email/password.
Lists all temperature/humidity sensors tied to the account.
Reads data from the device history endpoint for the requested period, starting from the device's creation date when no period is given.
Timestamps are parsed from high-precision strings.

Ubibot

Unified Endpoint Integration
Authenticates with username and SHA256-hashed password.
Lists all sensors and their fields.
Extracts temperature/humidity from structured feed data for the requested period.
Parses ISO-formatted timestamps.
Uses a wider 3-hour polling overlap than the other providers.

Victron

Unified Endpoint Integration
Authenticates with email/password.
Fetches installations and their system overviews.
Identifies temperature sensors based on device class.
Retrieves temperature data in 15-minute intervals.
Splits long periods into requests of at most 10,000 data points each.

Inficold

See the Inficold Integration Guide for setup, configuration and troubleshooting.

Devices are found through the user's Inficold dashboard, since the official API has no device list. Only the device the user selects is then checked through the official API, with the server-side token.
Each chamber (1 to 3 per device) is a separate source and cooling unit, identified as <device_id>:<chamber>, and is checked against the device's current chamber count.
Reads temperature and humidity for the requested period from the history endpoint, following every page. The API filters only by local date, so readings are filtered to the exact period after download.
Timestamps carry no timezone. They are interpreted in Asia/Kolkata and stored in UTC.
"NaN", missing and malformed values are skipped, and an empty successful response counts as a successful poll.
Device metadata (name, chamber count) is cached per device, with a stale fallback when Inficold is briefly unavailable.

Shared adapter contract

Every provider is an adapter class registered in services/sensors/registry.py, extending SensorIntegrationBase.

No exceptions: every adapter, Ecozen and Inficold included, follows the same contract.
Built from a plain configuration: each adapter receives a dictionary with type, username, password and source_id, and never reads Django models.
Runtime credentials: a saved integration becomes that configuration through resolve_sensor_credentials(). Providers that use a server credential add it through their runtime_credentials() hook, on every call.

Provider	`username`	`password`	`source_id`	Stored password
Figorr, Ubibot, Victron	Account username	Account password (decrypted)	Provider sensor ID	Encrypted account password
Ecozen	Machine ID	Runtime OAuth bearer token, issued from `ECOZEN_CLIENT_ID`/`ECOZEN_CLIENT_SECRET`	Machine ID	Empty
Inficold	Device ID	Server API token (`INFICOLD_API_TOKEN`)	`<device_id>:<chamber>`	Empty

The adapter methods are:

authorize(): prepares the provider authorization, such as a login, an OAuth token or a bearer header.
list_sources(): the sources available for the configuration, as {"id", "name"} items. For Ecozen and Inficold, these are the configured machine or the device's chambers, from a live check.
get_datums_from_source(source_id, since, until): readings for the period, as normalized SensorDatum values.
test_connection() and validate_source(source_id, live=True) (Ecozen and Inficold): provider-specific live validation methods. Ecozen uses these for setup connection testing; save verifies its signed source ticket without a live request, and polling retrieves data through get_datums_from_source(). Inficold performs live source validation during save; polling resolves its source using metadata that can be cached.

Each SensorDatum has:

value, with None meaning a missing reading.
datetime_stamp, the provider timestamp. Naive values are treated as UTC.
specification_type (TEMPERATURE or HUMIDITY) and unit (CELSIUS, FAHRENHEIT or PERCENT_RH).
An optional set_point_value.

The shared ingestion then normalizes and stores the datums (see Polling and ingestion).

Everything outside the adapter belongs to the application layer:

Portal discovery and setup-ticket helpers (module-level functions outside the adapter classes in ecozen.py and inficold.py).
Signed setup tickets.
User and company permission checks.
Persistence and polling state.

Adding a new provider means implementing authorize, list_sources and get_datums_from_source on a new adapter and registering it.

In summary, the sensor integration architecture unifies multiple platforms under a single adapter contract, with dedicated discovery and connection tests for Ecozen and Inficold, streamlining both frontend and backend interactions. This approach gives providers a common setup and ingestion contract. Ecozen's current concurrency and cooldown handling does not yet provide fleet-wide request pacing or fair scheduling; reliable polling at scale depends on fitting the confirmed provider quota.

2. Simulation Integration: COMSOL (Digital Twin)

In addition to real-time sensor data, Coldtivate runs predictive Digital Twin simulations using COMSOL. These simulations estimate how long produce will remain fresh under current storage conditions, enabling smart notifications, shelf-life tracking, and proactive interventions.

What It Does

Runs predictive simulations per crate using COMSOL models
Predicts:
- Remaining shelf life (shelf_life)
- Quality score (quality_dt)
- Simulated temperature (temperature_dt)
Updates crate records through a callback to the Base API

How It Works

Step	Action
1. Crate Selection	A scheduled Celery task (`recompute_digital_twin`) filters eligible crates that meet criteria such as weight, recent data, and enabled digital twin. Only one crate per produce is sent.
2. Trigger Microservice	The task sends a POST request to the `Comsol-Digital-Twins` microservice with parameters, temperature history, and a callback URL.
3. Job Queued	The job is enqueued by an internal queue manager. Duplicate jobs (same crate) are skipped.
4. FUSE-Controlled Input	The microservice writes simulation inputs to a FUSE-mounted file (`event_loop_control.txt`) which is monitored by COMSOL Runtime.
5. Simulation Execution	COMSOL runs the appropriate model based on `fruit_index` and writes results to output files through FUSE.
6. Output Parsing	FUSE handlers detect completion and extract results from `output_PL.txt` and `outputvalues.txt`.
7. Callback Dispatch	The microservice sends a JSON callback to the Base API with prediction outputs.
8. Database Update	The Base API receives the callback and updates both the corresponding crate and all the others that share the same produce id in PostgreSQL.

Models Used

The simulation model is selected based on the crop's digital_twin_identifier. Available models include:

DigitalTwinAppleV5.mph
DigitalTwinTomatoV5.mph
DigitalTwinPotatoV5.mph
DigitalTwinBananaV5.mph
DigitalTwin_GenericV5.mph (fallback model)

Each model is calibrated to reflect the spoilage dynamics of a specific crop.

Reliability Measures

Deduplication: Prevents enqueuing multiple jobs for the same crate.
Timeouts: COMSOL simulations are terminated after 2 minutes if unresponsive.
Failsafes: Errors are logged, email alerts are sent, and failures are gracefully skipped.
Monitoring: Print-based logs support integration with external monitoring tools.
Cleanup: All FUSE buffers are cleared after each job to manage memory.

Summary

By combining sensor-based data collection with simulation-based forecasting, Coldtivate offers a powerful infrastructure for managing storage conditions and extending the life of perishable goods.