Python Programming in 2023

For longtime readers here at House of Beor, you will know that back in the early 2000’s I was working with Plone and Python quite a bit. For a variety of reasons I stopped and got into PHP, Drupal, Obj-C, Swift and well you get the point. Here we are on the verge of 2023 and looks like Python is something I’ll be spending quite a bit of time with. Not my favorite language, but maybe things have improved and changed. Let’s see!

10/28/24: I’ve made a few updates to clarify use of pipenv, add some additional tips and corrected some grammatical issues. Added a reminder about where to pick the python runtime in MS Visual Studio Code.

Setting up Python and an IDE

As you all should know, my machine is a MacBook Pro 2019 running macOS Monterey. This presents special challenges for the Python developer because Apple still includes an ancient version of Python on the machine that you have to deal with. I do also have a 2017 MBP running macOS Ventura. I’ll attempt to make sure that all my suggestions work on both. There are many posts about how to install various version managers that are similar to the node version manager (nvm), but I’m not had too much luck with those working reliably on macOS.

I’ll probably get flack for this, but what I’ve done is use Homebrew to install python for me. I do not anticipate the need for multiple versions of python on my local machine. I will be using Python 3 and generally sticking to the latest version, which is 3.10.8 as I write this. Getting that working is very simple: “brew install python” and then follow the configuration suggestions.

Now for an integrated development environment (IDE)…there are some obvious choices with MS VS Code and Jetbrains Pycharm Community. Komodo recently went belly up so that one is off the table. This is a bit of a toss up. PyCharm Community has some limitations when it comes to Jupyter notebooks so for most folks I think VS Code is probably the way to go. This is largely a matter of personal preference. I see that JetBrains has just released a revamped UI for their IDEs so I think I am going to go with their IDEA, which includes Python plugins (so basically PyCharm Pro). I’ll go back and forth between VS Code and IDEA to ensure things work the same.

A temping thing to do at this point would be to create alias’ for python 3 to python 2. Do not do this! This will mess up how a virtual environment tool finds the correct python. See the next few steps…

Managing Python Environments

Although I will tend to stick with the latest Python 3 versions, for projects I will use an environment manager that will allow me to use different version of libraries and dependencies without having this collide. I looked for various options and it looks like pipenv is the way to go (check this post for more). Its similar to other environment managers and dependency managers that I have worked with so I feel pretty comfortable with it. A quick install and a update to my path in .zshrc so that the binaries can be found (instructions are in the install output) and you are good to go. Very nice indeed!

To create a new environment, create a directory, cd to that directory and then type: pipenv install

Can’t get much easier than that.

I suppose that image is so redacted its silly. Basically showing the commands to enter and exit pipnv’s environment from within VS Code’s terminal. To install a dependency just type (using fastapi as an example):

pipenv install fastapi

and pipenv will install it and add it to the local pipfile and pipfile.lock files. Always remember to exit your virtual environment to install a new dependency and then re-enter. To install all dependencies in the pipfile just use “pipenv install”.

To enter the virtual environment, use “pipenv shell” and to exit the environment use “exit” (not deactivate). If you did use deactivate by accident type exit again and you should be ok. In MS Visual Studio Code, you must select the pipenv installed python runtime. in the lower right hand corner of the GUI:

I did notice some additional “flakiness” with how dependencies are installed and resolved when using the terminal from within the IDEA interface. It can be confusing when you create a project from the IDEA project wizard since it will create a virtual environment for you in ~/.local/share/virtualenvs. So then if you are manipulating your project from within IDEA and the terminal things can get pretty confusing. Add on top if you also create projects outside of the IDE it can cause a migraine in 3.5 seconds. Same is true with VSC. Best to be very consistent with how you create your projects and interact with them.

It definitely helps to check “which python” to see which env you are using. Also keep an eye on what’s happening when you “activate” on “pyenv shell” into an env. Look at the paths!

Often I have to “deactivate” and then “exit” to get to the proper position in the hierarchy.

I’ve also found that you need to o into IntelliJ’s Settings –> Editor –> Inspections and search on requirements.txt and turn that off since I’m using the pipfile, not requirements.txt.

Things to Remember in Python

Constructors in Python are bizarre. As with PHP, a lot of Python’s OOP features are essentially bolt-ons ( maybe sidecar is a better term today. You’re just not cool unless you are talking in k8s, right?). Anyway the constructor’s format is:

class Animal:

    def __init__(self, age, height, weight):
        self.age = age
        self._height = height
        self.__weight = weight

In Python you have to use the self keyword to access your attributes within the class definition.

Private or Not So Private _ vs __. You can have private class members, but all members are public by default. Putting an underscore at the start of the name will make it semi-private (your IDE will not list it in the popup tips, but you can in fact still use it if you know its there. A double underscore will make it fully private. Example is in the code block above.

Type checking in Python is really only focused on passing and returning data from functions. After installing the typing dependency, ou can define a method signature, mark things as optional, and specify a return type. The following says I have a function that takes three parameters, one of which is optional, and returns a string.

def my_function(a: int, b:float, c: Optional[int]) -> String

If you implement type checking you can use the dependency “mypy” to check you implementation. Install that package and you can execute it from terminal to check a file: “mypy my_filename.py” and it will work like a linter to check and throw warnings. You can also use something like FastAPI in combination with pydantic and typing to bring additional type checking to your endpoints. Using pydantic‘s BaseModel as a class to inherit from will allow you to define types and have these bubble thru to FastAPI’s doc generation features.

Ont thing I noticed with pydantic is that you can assign an alias to a Field. However it is not an alias, but a complete replacement for the original name. You can’t use the original name anymore. I also found that this feature does not work reliably. Check out the screen shot below:

You can see my class definition in the bottom right. I have a field username with an alias “name”. The print statement shows that the name is correctly set in the user_content object that is passed to create the user. The next print statement shows that the alias was ignored and you see ‘username’ instead. So I think I’ll not use this specific feature for now. Maybe I’ll figure out I’m doing something wrong in a week or two…

Multiple Inheritance –> yes, multiple inheritance!

Class methods and variables are available before a class is instantiated. You can define a class with a set of functions and attributes that are static (outside the init block). These will be available without creating a class instance. Kinda interesting for how to get default configuration values into a program.

A major improvement in Python 3 is F strings. These really make formatting strings quite a bit easier for output to the console or log files.

Interesting Python Packages

• Requests: Make HTTP handling easier with this package.
• FastAPI: A lighweight application and API driven framework. Great for quick PoC’s.
• Flask/Quart/Jinja – the Pallets Projects: a set of packages for web application development that includes Flask and Quart.
• PyTorch: Machine learning framework, ecosystem, and associated tools.
• Pandas: de facto package for working with data structures and tabular data in python.
• SciKit: machine learning framework – yes there are a lot of these!
• NetworkX: Not networks like the interwebs, but networks like neural and graph
• Keras: modern and well documented deep learning framework
• TensorFlow: machine learning framework available for many different platforms.
• NLTK: natural language processing framework and tools.
• Dash: Plotly’s dash is a low code framework for data analysis and display in python.

I have not had a chance to look at all of these in depth, but they all look quite interesting, have great documentation/training materials, and have fun use cases for both personal and work situations.

Python Programing in 2023

There’s no doubt that Python is a strong language and pretty much unavoidable in the data science area. Although it is not my favorite language, like Ruby, it does have many features and functions that are very handy for processing text and data. There are also a ton of libraries and components to grab and build with. I’ll be spending a lot of time with Python and running up my HackerRank.com score. 🙂