Skip to content

Integrations & Data Collection

This document provides a guide to how the platform integrates with external systems to collect and process data. It covers both temperature sensor providers and simulation-based integrations like COMSOL, which work together to power Coldtivate’s core features around monitoring and shelf-life prediction.

1. Temperature Sensor Integrations

These integrations allow the platform to collect accurate, real-time temperature data from cold storage units.

Viewing temperature and humidity history

Operators, Registered Employees, and Site Officials can use the room conditions graph to review environmental readings for a cooling unit.

  1. Open Cooling Units and select Room Conditions.
  2. Select the cooling unit you want to review. Changing the cooling unit resets the graph to Temperature and the 24H range.
  3. Select Temperature or Humidity above the graph. Temperature values are shown in degrees Celsius (°C), while relative humidity values are shown as a percentage (%). If the selected cooling unit has humidity readings but no temperature readings, the view automatically opens the humidity graph.
  4. Press and move across the graph to inspect a reading. The value card above the graph shows the selected value, date, and time. When you release the graph, the card returns to the latest reading in the selected period.

The graph initially shows the latest value in the selected period. A line connects multiple readings; a period containing only one reading is shown as a single point. Pull down on the page to request the latest available data.

Selecting a time range

Use the tabs below the graph to control the period displayed:

Range Usage
24H The default view. Shows readings from the preceding 24 hours.
7D Shows readings from the preceding seven days.
CUSTOM Shows readings between a start and end date that you select. Both dates are required before data is loaded.

Custom ranges are limited to the last seven days:

  • The start and end dates cannot be in the future.
  • The date pickers do not allow dates earlier than seven days before the current date.
  • The selected period cannot exceed seven days. If it does, the app keeps the previous end date and asks you to choose a shorter period.
  • Changing the start date clears an existing end date when the resulting range would be invalid. Select a new end date to continue.

While data is being requested, wait for the loading state to finish before changing the metric or period again. If no readings exist for the selected metric and period, the graph displays No data available and identifies whether the empty period is the last 24 hours, the last seven days, or the custom period. Select another period to look for historical readings. A newly connected sensor may also need time to transmit its first reading.

Usage examples

  • Review a recent temperature change: Keep Temperature and 24H selected, then move across the graph to compare individual readings and their timestamps.
  • Check humidity conditions: Select Humidity. Confirm that the value is shown as a percentage, then review the graph for sustained high or low readings.
  • Investigate a known event: Select CUSTOM, choose the event date as the start date and an allowed end date, then inspect readings around the event time. Custom history is limited to the most recent seven days.

Responding to stale sensor data

For a cooling unit with a connected sensor, the page shows when the temperature data was last updated. The data is considered stale when the backend reports a sensor error or the latest temperature reading is more than 12 hours old.

When data is stale:

  • Last updated appears in red.
  • A red Stale badge with a clock warning icon appears below it.
  • Selecting the badge expands a Sensor data stale warning. When the last reading time is available, the warning reports approximately how many hours or days have passed since the update.

Do not treat the displayed value as a current room reading while the warning is present. Note the cooling unit and its Last updated time. Operators should notify a Registered Employee or the Site Official assigned to the cooling unit; Registered Employees and Site Officials should check the sensor connection and restore data transmission. Refresh the page after the connection is restored and confirm that the timestamp advances and the Stale badge disappears before relying on the reading as current.

Frontend Integration

  • Form Management:

All sensors are integrated through a form built with react-hook-form, ensuring robust state management, validation, and user-friendly input handling. Each form collects the necessary credentials specific to the chosen provider.

Ecozen, however, requires a separate form and backend endpoint due to its particularities.

  • API Connectivity:

The frontend is built with React (and React Native) and is responsible for collecting user input and passing sensor credentials to the backend. The updated process simplifies the user experience:

Sensor Setup Flow:

  • The user selects a sensor provider (Ecozen, Figorr, Ubibot, or Victron).

  • They fill in credentials (email/username and password).

  • The frontend calls a unified endpoint through a centralized SensorsService that abstracts API calls for validating the credentials and retrieve all sensors associated with the account.

  • The user selects a sensor from the returned list, which is saved upon cooling unit creation.

In summary, the frontend is responsible solely for gathering and transmitting sensor-specific data, while the backend takes care of establishing sensor connections and handling all subsequent operations. More on that, in the next section.

Backend Integration

The backend handles authentication, sensor listing, periodic temperature syncing, and error handling. Key aspects of the updated architecture include:

  • Unified Sensor Listing API:

  • A single endpoint (storage/v1/user-sensor/sources/) receives the sensor type (integration_type) and credentials (username, password).

  • After successful authentication, the backend returns a list of available sensors for the given account.

  • This list includes relevant metadata like the sensor ID, name, last seen timestamp, and specification type (e.g., temperature or humidity).

  • Ecozen Exception:
    • Ecozen is the only provider with a dedicated backend endpoint due to its unique API structure.
    • Endpoint: storage/v1/ecozen/test-connection
    • Payload: 
      {
  "username": "[email protected]",
  "password": "Hello123",
  "source_id": "machine_id_123"
}
    • Behavior:
    • Authenticates using the provided credentials.
    • Uses the source_id (which corresponds to the sensor's machine_id) to query Ecozen for temperature data.
    • If the authentication and sensor connection are successful, returns a 200 OK status.
    • On failure:
      • 401 for invalid credentials or missing token.
      • 404 for invalid machine ID.
      • 400 for generic or unexpected errors.
    • This response flow allows the frontend to confirm the sensor's availability before associating it with a cooling unit.

  • Scheduled Data Syncing:

  • The task update_temperature periodically fetches temperature data from all connected sensors.

  • Data is fetched via integration-specific classes (e.g., FigorrIntegration, UbibotIntegration, VictronIntegration, EcozenIntegration).

  • Data is stored in CoolingUnitSpecifications and used for operations like shelf-life predictions.

  • Error Handling & Notifications:

  • Failed data pulls result in system-generated SENSOR_ERROR notifications tied to the relevant cooling unit.

  • Once data retrieval resumes, the error notifications are automatically cleared.

  • Credential Storage:

  • Credentials are securely stored in the SensorIntegration model and used during each scheduled sync operation.

  • Sensor Deletion:

  • A DELETE endpoint allows removal of specific sensors if needed, with permission validation based on the user’s organization.

Sensor-Specific Behaviors

Below is a summary of each supported sensor type:

Ecozen

  • Dedicated Endpoint & Form
  • Authenticates with email/password, retrieves room and set point temperatures.
  • Timestamps are in milliseconds and parsed into datetime.
  • Readings are stored only if spaced apart by 15+ minutes.
  • API errors trigger sensor failure notifications.

Figorr

  • Unified Endpoint Integration
  • Authenticates with email/password.
  • Lists all temperature/humidity sensors tied to the account.
  • Reads data from the device history endpoint.
  • Timestamps are parsed from high-precision strings.
  • Values are recorded if newer than the last entry.

Ubibot

  • Unified Endpoint Integration
  • Authenticates with username and SHA256-hashed password.
  • Lists all sensors and their fields.
  • Extracts temperature/humidity from structured feed data.
  • Parses ISO-formatted timestamps.
  • Stores data only if valid and new.

Victron

  • Unified Endpoint Integration
  • Authenticates with email/password.
  • Fetches installations and their system overviews.
  • Identifies temperature sensors based on device class.
  • Retrieves temperature data in 15-minute intervals.
  • Caps retrieval to 10,000 data points per request.
  • Includes fallback for initial syncs when no DB entries exist.

Extensibility

The sensor integration system is designed for scale. Adding a new provider only requires implementing three base methods:

  • authorize
  • list_sources
  • get_datums_from_source

This ensures new sensors can be seamlessly onboarded following a consistent contract.

In summary, the sensor integration architecture unifies multiple platforms under a single API contract (except for Ecozen), streamlining both frontend and backend interactions. This approach simplifies sensor selection for users and ensures scalable, reliable data syncing for the platform.

2. Simulation Integration: COMSOL (Digital Twin)

In addition to real-time sensor data, Coldtivate runs predictive Digital Twin simulations using COMSOL. These simulations estimate how long produce will remain fresh under current storage conditions, enabling smart notifications, shelf-life tracking, and proactive interventions.

What It Does

  • Runs predictive simulations per crate using COMSOL models
  • Predicts:
    • Remaining shelf life (shelf_life)
    • Quality score (quality_dt)
    • Simulated temperature (temperature_dt)
  • Updates crate records through a callback to the Base API

How It Works

Step Action
1. Crate Selection A scheduled Celery task (recompute_digital_twin) filters eligible crates that meet criteria such as weight, recent data, and enabled digital twin. Only one crate per produce is sent.
2. Trigger Microservice The task sends a POST request to the Comsol-Digital-Twins microservice with parameters, temperature history, and a callback URL.
3. Job Queued The job is enqueued by an internal queue manager. Duplicate jobs (same crate) are skipped.
4. FUSE-Controlled Input The microservice writes simulation inputs to a FUSE-mounted file (event_loop_control.txt) which is monitored by COMSOL Runtime.
5. Simulation Execution COMSOL runs the appropriate model based on fruit_index and writes results to output files through FUSE.
6. Output Parsing FUSE handlers detect completion and extract results from output_PL.txt and outputvalues.txt.
7. Callback Dispatch The microservice sends a JSON callback to the Base API with prediction outputs.
8. Database Update The Base API receives the callback and updates both the corresponding crate and all the others that share the same produce id in PostgreSQL.

Models Used

The simulation model is selected based on the crop's digital_twin_identifier. Available models include:

  • DigitalTwinAppleV5.mph
  • DigitalTwinTomatoV5.mph
  • DigitalTwinPotatoV5.mph
  • DigitalTwinBananaV5.mph
  • DigitalTwin_GenericV5.mph (fallback model)

Each model is calibrated to reflect the spoilage dynamics of a specific crop.

Reliability Measures

  • Deduplication: Prevents enqueuing multiple jobs for the same crate.
  • Timeouts: COMSOL simulations are terminated after 2 minutes if unresponsive.
  • Failsafes: Errors are logged, email alerts are sent, and failures are gracefully skipped.
  • Monitoring: Print-based logs support integration with external monitoring tools.
  • Cleanup: All FUSE buffers are cleared after each job to manage memory.

Summary

By combining sensor-based data collection with simulation-based forecasting, Coldtivate offers a powerful infrastructure for managing storage conditions and extending the life of perishable goods.