The purpose of the GCP Deployment Request is The purpose of this document is to allow our client`s Data Science team access Boxalino datasets, for the goal of running jupyter/notebook processes in the designed anaconda environmentsprepare and deploy their models in a Google Cloud environment.
| Table of Contents |
|---|
Environment Details
...
Python 3.7
git
Anaconda3
pip / pip3
setuptools
papermill
jupyter
google-api-python-client & google SDK libraries
Steps
Make a GCP Project Deployment Request with the Required Information information.
a GCP project will be provided to the requestor
Set the billing account on the new project.
This is required in order to be able to use the GCP resources.
Prepare the Required Files (application structure)
Load the content in a GCS bucket from the project.
| Infonote |
|---|
Your user email (as the requester) will be given the Editor, Owner and Project Billing Manager role. Share access to other people who need access to the project. |
| Tip |
|---|
The application is launched in a VM Virtual Machine in the project. The commands from commands.txt are executed. Additionally, you can SSH on the VM and update/check content. |
...
. Additionally, you can SSH on the VM and update/check content. |
As the manager of the application, you are responsible for the VM state.
You can follow the provided practices on:
We, at Boxalino, will extend the services available for the Data Science needs (schedulers, instance management, etc). Make sure to review the documentation.
Integration Access
Because the application is launched in the scope of the project, the following Google Cloud tools can be used:
...
BigQuery Data Editor : <client>_lab, <client>_views
BigQuery Data Viewer : <client>_core, <client>_stage, <client>_reports, <client>_intelligence
1. Project Deploy
Required Information
...
| 1 | project name | as will appear in your project`s list |
| 2 | the requestor is the one managing the applications running on the project; this email will receive messages (alert and notifications) for when the project is ready to be used; | |
| 3 | client name | (also known as the Boxalino account name) this is to ensure the access to the views, core & reports datasets (https://boxalino.atlassian.net/wiki/spaces/BPKB/pages/303792129/GCP+Project+Deployment#BigQuery-Datasets-Access ) |
| 4 | optional; the labels are used as project meta-information. see Labels | |
| 5 | optional; by default, the requestor will have full access and can further share with others. see Permissions |
...
go to the Billing menu in GCP console or check the billing projects https://console.cloud.google.com/billing/projects
Identify the project and click on the 3 dots. Select “Change Billing”
3. From the appeared window, select the Billing Account on which the costs of the Application will be billed
...
| Code Block |
|---|
chmod -R 777 <home value from instance.txt>/*
python3 <home value from instance.txt>/my-python-application.py
papermill <home value from instance.txt>/process.ipynb <home value from instance.txt>/process-output.ipynb |
...
| Code Block |
|---|
sudo gsutil rsync -r gs://<BUCKET>/ <APPLICATION-PATH> |
| Note |
|---|
Replace <BUCKET> with your storage bucket name (where the application files have been loaded). Replace <APPLICATION-PATH> with the path to your application (default: /home/project-name). |
...