Untangle Python Spaghetti - Deep dive into environments and dependencies management

By Cheuk Ting Ho

Elevator Pitch

After learning doing in Python, we started multiple Python or Data Science projects. Dependency management becomes a skill that we need to avoid requirement conflicts amount projects. In this talk, we will learn how dependencies management tools work and how to choose the right one to use.

Description

In this talk, we will use venv, the environment managing tool that come with CPython, and conda, the tool that is popular among data science partitioners to decipher how environment and dependency management works. First, we will go through some basic knowledge of how Python “sees” your package and know where to import them when needed. Then, we will make the audience aware that, to avoid the requirement conflict amount different projects, we need a new set of environments for each project.

Then, we will see how tools like venv and conda can create a new set of environments. We will cover both the practical aspect - how to use the tools and which one to use, and the theoretical aspect - the inner working of the tools and why they are different. This will make the audience get familiar with some tools that are at their disposal and inspire them to start using them for their work.

This talk is for beginner Pythonistas or data scientists who started to use Python professionally and those who are curious to know how the environment and dependency management tools work.

Outline

  • Introduction (5 mins)
  • How Python sees your packages (5 mins)
  • Why we need separate environments (1 mins)
  • How venv create new environment (2 mins)
  • Why is conda different from venv (2 mins)
  • Comparing venv and conda - which one to use (5 mins)
  • Conclusion (5 mins)