Integration Strategies

Owned by Dana Negrescu
Last updated: Apr 26, 2024
3 min read

Use this page to get familiar with the data integration strategies and how they can assist your project needs.

If your data is integrated with the use of a Boxalino-provided service (or repository), the integration team must review the source and validate that the project data flow (SW6, Magento2, etc) complies with he platform practices for content update & access.

Any custom flow or data sources must be adjusted/extended/recognized by the integration team.

We are paying close attention to our integrator`s needs for extensibility.
Please ask for assistance or guidance when needed.

 

FULL DATA INTEGRATION

PURPOSE

During a full data integration process it is done a full rewrite of your data index. This means:

  1. Adding/removing attributes to your data (updating the schema)

  2. Adding new items to your collection (ex: new products, new blog posts, new campaigns, etc)

  3. Removing items from the collection (the items which are not part of the full - are removed from the index)

REQUIREMENTS

This process must be ran daily (once per day).

We recommend to schedule this event within your project flow after all other 3rd party systems updates were done (ex: after your daily price update, stock review, daily reindex, etc)

The full export for doc_order & doc_user is done once at the account kick-off. Once the historical load of this information is made available, the doc_order & doc_user can be executed daily in delta mode (for new & updated documents)

ENDPOINT

Please use this endpoint for all your full data integration requests:

https://boxalino-di-full-krceabfwya-ew.a.run.app

GCP RESOURCES

The full data exports contents are available in the dedicated Boxalino account resources for 30 days.

 

DELTA DATA INTEGRATION

PURPOSE

During a delta data integration process it is done a partial update of your index. This means:

  1. It is possible to entirely rewrite the information for the already available items (partial update)

  2. It is possible to add new items to the index

  3. It is not possible to add new attributes to your content (no schema update)

REQUIREMENTS

This process can be run as often as every 5 - 10 minutes*. For consistency purposes, it should not run concurrently with the daily full data update.

* interval depends on the account statistics

ENDPOINT

Please use this endpoint for all your delta data integration requests:

https://boxalino-di-delta-krceabfwya-ew.a.run.app

GCP RESOURCES

The delta data exports contents are available in the dedicated Boxalino account resources for 7 days.

 

INSTANT DATA INTEGRATION

PURPOSE

The purpose is to instant update essential content information (ex: stock, availability, price, category, etc) (under 40 sec). This means:

  1. It is possible to update partially an item (ex: a few select properties)

  2. It is not possible to add new attributes to your content (no schema update)

  3. It is not possible to add new items to your collection

REQUIREMENTS

This process can be run real-time, item-based.

ENDPOINT

Please use this endpoint for all your instant data integration requests:

https://boxalino-di-instant-update-krceabfwya-ew.a.run.app

GCP RESOURCES

The instant data exports contents are available in the dedicated Boxalino account resources for 1 days.

 

CUSTOM BIGQUERY CONTENT INTEGRATION

PURPOSE

The purpose is to expose your custom BigQuery content to all Boxalino processing events:

  1. Data used to create custom facet options

  2. Data used to access other relevant details for views or lab processing

REQUIREMENTS

Share your datasets access with the Boxalino Service Account emails:

  1. 55483703770-compute@developer.gserviceaccount.com

  2. 285267441728-compute@developer.gserviceaccount.com

  3. Access to dataform processing (part of daily/full/delta processing) service-285267441728@gcp-sa-dataform.iam.gserviceaccount.com

  4. (deprecated) Access to daily processing: 473358938625-compute@developer.gserviceaccount.com

  5. (deprecated) Access to full processing: 748704719320-compute@developer.gserviceaccount.com

  6. (deprecated) Access to delta processing : 178258671205-compute@developer.gserviceaccount.com

  7. (deprecated) Access for reports processing: connection-daily@reports-290318.iam.gserviceaccount.com

  8. (deprecated) Access for lab processing: connection-daily@lab-daily.iam.gserviceaccount.com

  9. (optional) Access to mailchimp processing: 44011570818-compute@developer.gserviceaccount.com

  10. (optional) Access for p13nsync processing: 57928973091-compute@developer.gserviceaccount.com

Provide a read access on all relevant datasets in your custom GCP projects.