Versions Compared

Old Version 3

changes.mady.by.user Dana Negrescu

Saved on

compared with

New Version 4

changes.mady.by.user Dana Negrescu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Info

instance.txt, requirements.txt and commands.txt must end with an empty line (they are parsed dynamically)

instance.txt

The rows nr. 1-4 are required; the others are displayed with the default values (can be rewritten with the file)

Code Block
instance-name:application-name
machine-type:e2-micro
root-dir:\/srv\/app
email-to:requester@email.com
image-family:ubuntu-2004-lts
boot-disk-size:30
zone:europe-west1-b

requirements.txt

This is a typical requirements.txt file for a jupyter/python application.

Info

  • If your setup has been tested locally, you can save all the packages in the file with 
    $pip freeze > requirements.txt. 
    Keep in mind that in this case, requirements.txt file will list all packages that have been installed in virtual environment, regardless of where they came from

They will be installed on the conda environment.

Code Block
google-api-core==1.20.0
google-api-python-client==1.9.3
google-auth==1.17.2
google-auth-httplib2==0.0.3
[..]
google-cloud-core==1.3.0
google-cloud-storage==1.29.0
google-pasta==0.2.0

commands.txt

You can include the commands to run the jupyter process on application launch.

You can also include other commands same as the application has been tested on the local environment.

Code Block
papermill PROJECT-ROOT/process.ipynb PROJECT-ROOT/process-output.ipynb

env.yml

The env.yml file is used in the setup step in order to create the anaconda environment.

Code Block
name: gcp-application-name
channels:
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - ca-certificates=2020.1.1=0
  - certifi=2020.4.5.1=py37_0
  - ld_impl_linux-64=2.33.1=h53a641e_7
  - libedit=3.1.20181209=hc058e9b_0
  - libffi=3.3=he6710b0_1
  - libgcc-ng=9.1.0=hdf63c60_0
  - libstdcxx-ng=9.1.0=hdf63c60_0
  - ncurses=6.2=he6710b0_1
  - openssl=1.1.1g=h7b6447c_0
  - pip=20.0.2=py37_3
  - python=3.7.7=hcff3b4d_5
  - readline=8.0=h7b6447c_0
  - setuptools=47.1.1=py37_0
  - sqlite=3.31.1=h62c20be_1
  - tk=8.6.8=hbc83047_0
  - wheel=0.34.2=py37_0
  - xz=5.2.5=h7b6447c_0
  - zlib=1.2.11=h7b6447c_3
  - pip:
      - absl-py==0.9.0
      - ansiwrap==0.8.4
      - appdirs==1.4.4
      - astunparse==1.6.3
      - async-generator==1.10
      - attrs==19.3.0
      - backcall==0.2.0
      - black==19.10b0
      - bleach==3.1.5
      - cachetools==4.1.0
      - chardet==3.0.4
      - click==7.1.2
      - cycler==0.10.0
      - cython==0.29.20
      - dataclasses==0.6
      - decorator==4.4.2
      - defusedxml==0.6.0
      - dill==0.3.1.1
      - dm-tree==0.1.5
      - entrypoints==0.3
      - future==0.18.2
      - gast==0.3.3
      - gin-config==0.3.0
      - google-api-core==1.22.2
      - google-api-python-client==1.9.3
      - google-auth==1.17.2
      - google-auth-httplib2==0.0.3
      - google-auth-oauthlib==0.4.1
      - google-cloud-bigquery==1.25.0
      - google-cloud-bigquery-storage==1.0.0
      - google-cloud-core==1.3.0
      - google-cloud-storage==1.29.0
      - google-pasta==0.2.0
      - google-resumable-media==0.5.1
      - googleapis-common-protos==1.52.0
      - grpcio==1.29.0
      - h5py==2.10.0
      - httplib2==0.18.1
      - idna==2.9
      - importlib-metadata==1.6.1
      - ipykernel==5.3.0
      - ipython==7.15.0
      - ipython-genutils==0.2.0
      - ipywidgets==7.5.1
      - jedi==0.17.0
      - jinja2==2.11.2
      - jsonschema==3.2.0
      - jupyter==1.0.0
      - jupyter-client==6.1.3
      - jupyter-console==6.1.0
      - jupyter-core==4.6.3
      - kaggle==1.5.6
      - keras==2.3.1
      - keras-applications==1.0.8
      - keras-preprocessing==1.1.2
      - kiwisolver==1.2.0
      - markdown==3.2.2
      - markupsafe==1.1.1
      - matplotlib==3.2.1
      - mistune==0.8.4
      - nbclient==0.4.0
      - nbconvert==5.6.1
      - nbformat==5.0.7
      - nest-asyncio==1.3.3
      - notebook==6.0.3
      - numpy==1.18.5
      - oauth2client==4.1.3
      - oauthlib==3.1.0
      - opencv-python-headless==4.2.0.34
      - opt-einsum==3.2.1
      - packaging==20.4
      - pandas==1.0.4
      - pandas-gbq==0.13.2
      - pandocfilters==1.4.2
      - papermill==2.1.2
      - parso==0.7.0
      - pathspec==0.8.0
      - pexpect==4.8.0
      - pickleshare==0.7.5
      - pillow==7.1.2
      - prometheus-client==0.8.0
      - promise==2.3
      - prompt-toolkit==3.0.5
      - protobuf==3.8.0
      - psutil==5.7.0
      - ptyprocess==0.6.0
      - py-cpuinfo==6.0.0
      - pyarrow==0.17.1
      - pyasn1==0.4.8
      - pyasn1-modules==0.2.8
      - pydata-google-auth==1.1.0
      - pygments==2.6.1
      - pyparsing==2.4.7
      - pyrsistent==0.16.0
      - python-dateutil==2.8.1
      - python-slugify==4.0.0
      - pytz==2020.1
      - pyyaml==5.3.1
      - pyzmq==19.0.1
      - qtconsole==4.7.4
      - qtpy==1.9.0
      - regex==2020.6.8
      - requests==2.23.0
      - requests-oauthlib==1.3.0
      - rsa==4.6
      - scipy==1.4.1
      - send2trash==1.5.0
      - sentencepiece==0.1.91
      - six==1.15.0
      - tb-nightly==2.3.0a20200615
      - tenacity==6.2.0
      - tensorboard-plugin-wit==1.6.0.post3
      - tensorflow-addons==0.10.0
      - tensorflow-datasets==3.1.0
      - tensorflow-hub==0.8.0
      - tensorflow-metadata==0.22.2
      - tensorflow-model-optimization==0.3.0
      - termcolor==1.1.0
      - terminado==0.8.3
      - testpath==0.4.4
      - text-unidecode==1.3
      - textwrap3==0.9.2
      - tf-estimator-nightly==2.3.0.dev2020061501
      - tf-models-nightly==2.2.0.dev20200615
      - tf-nightly==2.3.0.dev20200615
      - tf-slim==1.1.0
      - toml==0.10.1
      - tornado==6.0.4
      - tqdm==4.46.1
      - traitlets==4.3.3
      - typed-ast==1.4.1
      - typeguard==2.9.1
      - typing==3.7.4.1
      - uritemplate==3.0.1
      - urllib3==1.24.3
      - wcwidth==0.2.4
      - webencodings==0.5.1
      - werkzeug==1.0.1
      - widgetsnbextension==3.5.1
      - wrapt==1.12.1
      - zipp==3.1.0
prefix: /opt/conda/envs/gcp-application-env

Application Launch

Once the Required Files are uploaded in a GCS bucket from the project, contact Boxalino.
Provide the following information:

...