Enabling effective data science through lifecycle rules in SageMaker Studio

Architecture

Lifecycle rules

Kernel shutdown (DO THIS FIRST)

import base64
import boto3
session = boto3.Session()
client = session.client('sagemaker')
# taken from: sagemaker-studio-lifecycle-config-examples/on-jupyter-server-start.sh at main · aws-samples/sagemaker-studio-lifecycle-config-examples · GitHub
extension_install_script = '''
set -eux
sudo yum -y install wget# Saving the tarball to a file or folder with a '.' prefix will prevent it from cluttering up users' file tree views:
mkdir -p .auto-shutdown
wget -O .auto-shutdown/extension.tar.gz https://github.com/aws-samples/sagemaker-studio-auto-shutdown-extension/raw/main/sagemaker_studio_autoshutdown-0.1.5.tar.gz
pip install .auto-shutdown/extension.tar.gz
jlpm config set cache-folder /tmp/yarncache
jupyter lab build --debug --minimize=False
# restarts jupyter server
nohup supervisorctl -c /etc/supervisor/conf.d/supervisord.conf restart jupyterlabserver
'''
domain_id = client.list_domains()['Domains'][0]['DomainId']studio_lifecycle_config_response = client.create_studio_lifecycle_config(
StudioLifecycleConfigName='sagemaker-shutdown-extension’, StudioLifecycleConfigContent=base64.b64encode(extension_install_script.encode('ascii')).decode('utf-8'),
StudioLifecycleConfigAppType='JupyterServer',
)
client.update_domain(
DomainId=domain_id,
DefaultUserSettings={
'JupyterServerAppSettings' : {
'DefaultResourceSpec': {
'LifecycleConfigArn' : studio_lifecycle_config_response['StudioLifecycleConfigArn'],
'InstanceType': 'system'
},
'LifecycleConfigArns' : [
studio_lifecycle_config_response['StudioLifecycleConfigArn']
]
}
}
)

Environment management

# From JupyterServer
mkdir conda
conda config --add envs_dirs /root/conda
conda config --add envs_dirs /home/sagemaker-user/conda
# Important – Run inside image terminal in kernel
conda create --name myenv python=3.7.10 -y # defaults to new environment
conda activate myenv
pip install ipykernel
# Restart kernel from the Jupyter Notebook interface

Conclusion

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store