Import / Export
ICC can reason about the live data of a running application to calculate optimizations. This data is usually taken from production environments because we want to optimize the “real” system. The data can also be processed on other environments to elaborate and test the ICC optimizations.
Currently ICC supports the import/export to a S3-compatible bucket API, though ICC has been designed to quickly support more storage systems if necessary. When exported, data is signed using hmac-sha256 and the signature is verified when imported.
Configuration
Section titled “Configuration”An ICC instance can be configured as an importer or exporter depending on thePLT_RISK_COLD_STORAGE_EXPORTER configuration:
| PLT_RISK_COLD_STORAGE_EXPORTER | ICC role |
|---|---|
| true | export |
| false | import |
| [not specified] | disabled (no import or export, useful in dev / local env / demo). |
PLT_RISK_COLD_STORAGE_EXPORTER=truePLT_RISK_COLD_STORAGE_DROPOFF_TYPE=s3
PLT_RISK_COLD_STORAGE_AWS_ACCESS_KEY_ID=PLT_RISK_COLD_STORAGE_AWS_SECRET_ACCESS_KEY=PLT_RISK_COLD_STORAGE_AWS_REGION=us-west-1PLT_RISK_COLD_STORAGE_BUNDLE_BUCKET=plt-risk-bucketIn a standard ICC configuration, the production environment will be the only exporter and pre-production environments will act as importers.
HMAC signature
Section titled “HMAC signature”Data is signed using hmac-sha256 (https://en.wikipedia.org/wiki/HMAC). HMAC can provide authentication using a shared secret, which must be the same on exporter/importers. e.g.:
"PLT_RISK_COLD_STORAGE_HMAC_SECRET_KEY": "mysecretkey"ICC as Exporter
Section titled “ICC as Exporter”Example of ICC configured as Exporter:
"PLT_RISK_COLD_STORAGE_EXPORTER": "true","PLT_RISK_COLD_STORAGE_EXPORTER_DROPOFF_TYPE": "s3","PLT_RISK_COLD_STORAGE_AWS_ACCESS_KEY_ID": "yyyyyy","PLT_RISK_COLD_STORAGE_AWS_SECRET_ACCESS_KEY": "xxxxxxx","PLT_RISK_COLD_STORAGE_AWS_REGION": "eu-west-1","PLT_RISK_COLD_STORAGE_AWS_BUCKET": "icc-bucket","PLT_RISK_COLD_STORAGE_HMAC_SECRET_KEY": "mysecretkey"With this configuration, the “Export” setting will appear:
Export (and Import) frequency can be set using the selectors:
Once the export starts (as per schedule frequency or “run now”), the export appears on the list:
The detail of the export is also available:
ICC as Importer
Section titled “ICC as Importer”Changing the configuration in this way (it’s the same as exporter, but with PLT_RISK_COLD_STORAGE_EXPORTER to false):
"PLT_RISK_COLD_STORAGE_EXPORTER": "false","PLT_RISK_COLD_STORAGE_EXPORTER_DROPOFF_TYPE": "s3","PLT_RISK_COLD_STORAGE_AWS_ACCESS_KEY_ID": "yyyyyy","PLT_RISK_COLD_STORAGE_AWS_SECRET_ACCESS_KEY": "xxxxxxx","PLT_RISK_COLD_STORAGE_AWS_REGION": "eu-west-1","PLT_RISK_COLD_STORAGE_AWS_BUCKET": "icc-bucket","PLT_RISK_COLD_STORAGE_HMAC_SECRET_KEY": "mysecretkey"This will enable the import:
