In order to assist with the Data Integration (DI) efforts with 3rd party systems (ex: graphQL, Prismic, GraphCMS, BigQuery, PlentyMarkets, GCS, Contentful, etc), Boxalino is presenting the Data Transformer service https://boxalino-di-transformer-krceabfwya-ew.a.run.app/
...
Review the Transformer services:
is your connector type is not yet supported, please get in touch with Boxalino.
Use the https://boxalino-di-transformer-krceabfwya-ew.a.run.app/transformer/view service to identify the body content requested for the API request.
Identify the content types (blog, banner, categories, etc) that must be integrated from your 3rd party source.
Prepare the connector options https://boxalino.atlassian.net/wiki/spaces/BPKB/pages/662536193/Transformer#CONNECTOR with content from your 3rd party system.
Do a test connector request https://boxalino.atlassian.net/wiki/spaces/BPKB/pages/662536193/Transformer#TEST-REQUEST
Prepare the mappinghttps://boxalino.atlassian.net/wiki/spaces/BPKB/pages/662536193/Transformer#MAPPING for each content type you want to export.
Do a test sample request https://boxalino.atlassian.net/wiki/spaces/BPKB/pages/662536193/Transformer#REQUEST-DEFINITION
...
Endpoint | full data sync | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 |
| delta data sync | TBC | 2 | instant data sync | TBC | 3 | Action | /sync | ||
| 42 | Method | POST | |||||||||
| 53 | Body | use the public endpoint | |||||||||
| 64 | Headers | Authorization | Basic base64<<DATASYNC API key : DATASYNC API Secret>>  | ||||||||
| 75 |
| Content-Type | application/json | ||||||||
| 86 |
| client | account name | ||||||||
| 97 |
| mode | data sync mode: F for full, D for delta | ||||||||
| 108 |
| type | product, user, content, user_content, order. if left empty - it will check for all tables with the given tm | ||||||||
| 119 |
| tm | (optional) time , in format: YmdHis technical: used to identify the documents version | ||||||||
| 1210 |
| ts | (optional) timestamp, must be millisecond based in UNIX timestamp | ||||||||
| 1311 | dev | (optional) use this to mark the content for the dev data index | |||||||||
...
The connector properties are:
type (select from the available optionsprismic, graphcms, strapi, gcs, plentymarket, sftp)
options (gets generated based on the selected connector type)
...
| Expand | ||||
|---|---|---|---|---|
| ||||
The query content must be a list of filters and it should be escaped (copy-paste each [] filter into an escape/unescape online tool) Review this sample from Boxalino`s public Prismic repository:
|
| Expand | ||||
|---|---|---|---|---|
| ||||
The fields and filters must be escaped (copy-paste the graphiQL section into an escape/unescape online tool)
The filters are optional. |
...
do a
test/samplerequest without anymappingdeclaration to identify the default mappingsIn the case of prismic integrations, the following mappings are added by default:
Code Block "id": "id", "type": "type", "creation": "first_publication_date", "last_update": "last_publication_date", "link": "uid", "string_attributes" : {"linked_ids":"alternate_languages[*].id"}, "localized_string_attributes" : ["slugs", "href"], "raw" : { "localized_string_attributes" : [ "alternate_languages", "linked_documents"
] }
b. In the case of graphCms integrations, the following mappings are added by default:
| Code Block |
|---|
"id": "id", "type": "__typename", "creation": "createdAt", "last_update": "updatedAt", "persona_type": "createdBy.name", "persona_id": "createdBy.id", "localized_string_attributes" : ["stage"], "localized_datetime_attributes" : {"published_at":"publishedAt"} |
e) export of properties as raw JSON - use the raw node in the mapping
properties will be available as
bxraw_<repository <property name>this is helpful useful for exporting slices and other properties headless CMS specific content types that would require specific rendering in your template (ex: slices, sections, etc)
f) export linked-content : either using the contents property from the Data Structure or by exporting it as raw JSON
the mapping for
contentsis done in the following way:"contents":{"<source for input>":{"data-structure field":"prismic CMS field"}}"
g) in order to establish in which languages the content is available, a “language status” property is added dynamically ( value 0/1 based on language)
...
| Expand | ||
|---|---|---|
| ||
Warning : the mapping fields must exist in the connector options ~fields~ property.
|
| Info |
|---|
At Boxalino, we try to map the way our client expects for a headless integration to work. For this reason, if your connector integration libraries allows for content to be accessed as HTML or text (ex: data.content) - this is as well possible.
|
...
This must be declared in order to map the exported language (ex: “de”) with your connector export defined value (ex:”en-de”).
| Code Block |
|---|
"languages": {
"de": "en-de",
"fr": "fr-ch"
} |
The languages list is used in order to load the data from the headless CMS (per language). Further it will identify which ID from locale A is linked to which ID from locale B in order to generate localized data.
AVAILABLE SERVICES
In order to allow our integrators to validate that the existing data available in the 3rd party systems is properly exported, a few helper services are available:
...
You can also use the SAMPLE trigger in order to access one element of the given requested types.
The JSON response will be the JSONL structure as it is exported to Boxalino GCS.
...
| Expand | |||||
|---|---|---|---|---|---|
| |||||
|
...