For some reason I felt an urge to move from
envsubst to a python based templating system.
Enter pydantic. A fantastic package, which incidentally also can be used for settings management.
I wanted to split my configs into logical chunks but still have one monolithic
.env file. This is a no-brainer using pydantic
from pydantic import BaseSettings class GlobalSettings(BaseSettings): class Config(): env_file: str = ".env" class DbSettings(GlobalSettings): user: str password:str database: str . . . class Settings(GlobalSettings): something: str other: str dbsettings: DbSettings = DbSettings()
Imagine my surprise when I ran this code in ipython
Python 3.10.6 (main, Aug 2 2022, 00:00:00) [GCC 12.1.1 20220507 (Red Hat 12.1.1-1)] Type 'copyright', 'credits' or 'license' for more information IPython 8.4.0 -- An enhanced Interactive Python. Type '?' for help. In : from settings import Settings In : s = Settings() In : s.user Out: '👨 #(whoami)'
My surprise subsided quickly when I realized that I could have read the documentation
Even when using a dotenv file, pydantic will still read environment variables as well as the dotenv file, environment variables will always take priority over values loaded from a dotenv file..
It became adamant to find a way to not "pollute" my settings with all the weirdness of my environment. Turns out it's a pretty easy fix. It might even be a fix that won't break pydantic on the next upgrade [¹]
from pydantic.env_settings import EnvSettingsSource, SettingsSourceCallable class IgnoreLocalEnvSettingsSource(EnvSettingsSource): """Exists so that we can ignore local environment.""" def __call__(self, settings: BaseSettings) -> Dict[str, Any]: # noqa C901 """ Build environment variables suitable for passing to the Model. """ os.environ.clear() return super().__call__(settings) class GlobalSettings(BaseSettings): """Set the base config .env file.""" class Config: """Base config.""" env_file: str = ".env" env_file_encoding: str = "utf-8" def customise_sources( cls, init_settings: SettingsSourceCallable, env_settings: SettingsSourceCallable, file_secret_settings: SettingsSourceCallable, ) -> Tuple[SettingsSourceCallable, ...]: """Ignore __init__.""" return ( init_settings, IgnoreLocalEnvSettingsSource( env_file=cls.env_file, env_file_encoding=cls.env_file_encoding ), file_secret_settings, )
How useful was this exercise? In a production environment, probably not that much. Yes, there are (numerous) attack vectors that relies on replacing and overwriting environment variables. It's not inconceivable that an attacker might be able to exploit that.
However, in a development environment, like my local machine, there are a plethora of weird and forgotten environment variables. Knowing that the local env is not overwriting the variables in your .env file is one thing less to worry about.
Will this have adverse side effects? That remains to be seen, but if I understand the (pydantic) code correctly it shouldn't.
As an added bonus I got to clean up my local environment. My installation is only 6 months old, and still it was quite messy.
[¹]: I was looking at the github code thinking about implementing some really fancy stuff (I usually traverse down that particular rabbit hole before coming to my senses) when I realized that the pydantic I was looking at was nothing like the one on pypi.. which forced me to implement a much simpler solution.