![]() docker run -it -p 8888:8888 itsjafer/sparkmonitor This docker image has pyspark and several other related packages installed alongside the sparkmonitor extension. Quick Start To do a quick test of the extension Support for multiple SparkSessions (default port is 4040).For a detailed list of features see the use case notebooks.A notebook server extension that proxies the Spark UI and displays it in an iframe popup for more details.A graph showing number of active tasks & executor cores vs time. ![]() A timeline which shows jobs, stages, and tasks.A table of jobs and stages with progressbars.Automatically displays a live monitoring tool below cells that run Spark jobs in a Jupyter notebook.pyspark 3.X.X or newer (For compatibility with older pyspark versions, use jupyterlab-sparkmonitor 3.X).The extension provides several features to monitor and debug a Spark job from within the notebook interface itself. SparkMonitor is an extension for Jupyter Lab that enables the live monitoring of Apache Spark Jobs spawned from a notebook. Check his website out here.Īs a part of my internship as a Software Engineer at Yelp, I created this fork to update the extension to be compatible with JupyterLab - Yelp's choice for sharing and collaborating on notebooks. This project was originally written by krishnan-r as a Google Summer of Code project for Jupyter Notebook. Spark Monitor - An extension for Jupyter Lab
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |