docs.pelican.bernatchez.net - python

Configurar un índice de paquetes privados de proyecto

2025-05-24T18:23:52+00:00

Descripción general

Este documento describe la configuración de un índice de paquetes privados como una forma más conveniente de distribuir paquetes de Python.

La necesidad de usar algo como esto crece a medida que agregamos mantenedores y paquetes para mantener.

Nuestros paquetes de Python deben ser compatiles con el Pypi estándar de Pypi Index Pypi.

Utilizamos un índice de paquete privado hasta que estemos listos para hacer que un paquete esté disponible para el público en general, Luego cambiamos a Pypi.

Esta herramienta que estamos utilizando mantiene nuestros paquetes compatibles con PYPI para cuando la ponemos a disposición allí.

devpi

Es un sistema de índice de paquetes conforme.

We will use it to serve our private package index.

Configuring the SW for devpi

Este cómo se deriva del siguiente tutorial:

<https://devpi.net/docs/devpi/devpi/stable/+doc/quickstart-releaseprocess.html> _

We are working with python3. Queremos configurar un entorno virtual que se ajuste a nuestros supuestos y luego instalar dentro de eso. Llamamos al entorno devpi_venv. Creamos un directorio para nuestra configuración primero.

mkdir ${HOME}/devpi_setup
cd ${HOME}/devpi_setup

Instalamos los elementos esenciales desnudos.

python3 -m venv devpi_venv
source devpi_venv/bin/activate
python -m pip install -U pip
pip install wheel
pip install setuptools
pip install twine

Restview es un visor de formato de texto reestructurado que encuentro útil.(opcional)

pip install restview

Usted lo invoca y sigue reenviando su archivo a medida que lo modifica.

Como esto:

restview README.rst

rst2pdf es conveniente para generar copias PDF de .rst documuents.

pip install rst2pdf
rst2pdf doc/privatepypi.rst -o reformatted_docs/privatepypi.pdf

Configurar el servidor DevPI en nuestra computadora portátil

Instale el cliente Devpi y el servidor web El servidor no web, DevPI -servidor, se instala como consecuencia.

pip install -U devpi-web devpi-client

Inicializar un índice vacío (AT /VAR /DevPI-Merver de forma predeterminada) Tenga en cuenta que tenía un problema de permisos con el init. Así que primero hago esto manualmente:

sudo mkdir  /var/devpi-server
sudo chown ubuntu:ubuntu /var/devpi-server

devpi-init

Queremos que use un archivo de configuración para Supervisor Daemon. Esto generará un montón de archivos de configuración en el actual Directorio, el que queremos incluir.

cd ${HOME}/devpi_setup
devpi-gen-config

Esto nos da el archivo gen-config/supervisor-devpi.conf Copiar a/etc/supervisor/conf para que podamos comenzar arriba del servidor.Empecé editándolo para decir inicio = falso para que deba comenzar arriba manualmente.Luego lo copié al lugar correcto para ello.

sudo cp gen-config/supervisor-devpi.conf /etc/supervisor/conf.d/devpi-server.conf
sudo supervisorctl update
sudo supervisorctl start devpi-server

Crear un usuario, iniciar sesión como él y crear el índice 'Dev'

devpi user -c pbernatchez password=foobar
devpi login pbernatchez --password=foobar
devpi index -c dev bases=root/pypi

Utilice nuestro índice de desarrollo

devpi use testuser/dev

Ahora podemos hacer uso del índice privado.

Estamos utilizando Flit para publicar en nuestro índice y confía en el archivo: ~/.pypirc.

Entonces hacemos una entrada allí para nuestro índice.Le di el nombre 'mypypi':

[distutils]
index-servers =
   mypypi
   testpypi

[mypypi]
repository = http://localhost:3141/pbernatchez/dev
username = pbernatchez

[testpypi]
repository = https://test.pypi.org/legacy/
username = pbernatchez

De aquí en adelante, usando Flit, podemos referirlo como 'mypypi'.

deactivate
cd /home/ubuntu/allrepos/animbboard
source ${HOME}/venv_animbboard/bin/activate
flit build
flit publish --repository mypypi
pip uninstall  animbboard
pip install -i http://localhost:3141/pbernatchez/dev  animbboard

Set up a private package index

2025-05-24T18:23:52+00:00

Overview

At this writing our current practise is to generate versioned tarballs and to 'pip install' from those.

This describes setting up a private package index as a more convenient alternative to using tarballs directly.

Our python software needs to be kept compatible with the standard python package index for when we desire to offer it to the general public.

To ensure we are staying compatible with pypi, we need to run it through the process, but with a conforming private packaging index.

devpi to the rescue ('https://www.devpi.net/') .

It is a conforming package index system we can use to serve our private packages whithout releasing them to the public.

Configuring the SW for devpi

This information is derived from the following tutorial:

<https://devpi.net/docs/devpi/devpi/stable/+doc/quickstart-releaseprocess.html>_

We start by creating a virtual environment called "venv_devpi" by following the steps described in the document virtualenv.

Then we activate that environment and continue with the steps below.

Set up devpi server on our laptop

Install a devpi client and web server
pip install -U devpi-web devpi-client
Note:
The non-web server - devpi-server - is also installed as a side effect.
We must avoid needing sudo to access the index, so instead we change ownership from root to the logged in user. We also need to make sure supervisor runs our server as that user.

First manually do this:
sudo mkdir -p /var/devpi-server
sudo chown -R $(logname):$(logname) /var/devpi-server
ls -ld /var/devpi-server
Initialize an empty index (at /var/devpi-server by default)
devpi-init
Make a place to put our configuration data files:
mkdir -p ${HOME}/devpi_setup
The following will generate a bunch of config files, one of which we will need, under the current directory. Putting that stuff in ${HOME}/devpi_setup.
pushd . >/dev/null
cd ${HOME}/devpi_setup
devpi-gen-config
popd  >/dev/null
Note:
This gave us the file  ${HOME}/devpi_setup/gen-config/supervisor-devpi.conf
as a config file to start up our pypi server.  Start by editing
it manually to say autostart=False so that it will need to be
started up manually.
Append a line like "user = username" to the configuration so it launches as that user, the same user as above.
echo user = $(logname) >>${HOME}/devpi_setup/gen-config/supervisor-devpi.conf
Copy the config where where it needs to be, then start our server.
sudo cp ${HOME}/devpi_setup/gen-config/supervisor-devpi.conf /etc/supervisor/conf.d/supervisor-devpi.conf
sudo supervisorctl update
sudo supervisorctl start devpi-server
Create a user (within the server), login as that user and create the 'bzdev' index
devpi use http://localhost:3141
devpi user -c pbernatchez password=dummy
devpi login pbernatchez --password=dummy
devpi index -c bzdev bases=root/pypi
Use our bzdev index
devpi use pbernatchez/bzdev
Now we can make use of our private index.

We are using flit to publish to our index and it relies on the file : ~/.pypirc.

For pip I named it 'bzdevpi'. So we make an entry there for our index:
[distutils]
index-servers =
   bzdevpi
   testpypi

[bzdevpi]
repository = http://localhost:3141/pbernatchez/bzdev
username = pbernatchez

[testpypi]
repository = https://test.pypi.org/legacy/
username = pbernatchez
From here on, using flit,

To publish,

flit publish --repository bzdevpi

To install,

pip install -i http://localhost:3141/pbernatchez/bzdev foobar

Example:
deactivate
cd /home/ubuntu/allrepos/wait_ssh
source ${HOME}/venv_generic/bin/activate
flit build
flit publish --repository bzdevpi
pip uninstall wait_ssh
pip install -i http://localhost:3141/pbernatchez/bzdev wait_ssh

Configurar un entorno virtual de Python

2025-05-24T18:23:52+00:00

Asegúrese de que la versión del sistema Ubuntu sea lo suficientemente reciente

lsb_release -a

Impresión (o arriba)

Distributor ID:     Ubuntu

Description:        Ubuntu 24.04.3 LTS

Release:            24.04

Codename:           noble

python3 -V

Impresión

::: Python 3.12.3

Crear una base para todos nuestros entornos virtuales

mkdir ${HOME}/.venv
# Entonces podemos crear todos nuestros entornos virtuales en ${HOME}/.venv/

Crear nuestro entorno virtual

# Aqui nombramos el entorno virtual devpi
cd ${HOME}/.venv/
python3 -m venv devpi

O utilizando venvwrapper

mkvenv devpi

Una vez creado nuestro entorno, podemos instalar los elementos necesarios para empezar.

Actívalo

source ${HOME}/.venv/devpi/bin/activate

O utilizando venvwrapper

venv devpi

Instalar lo básico

# Instalar
python -m pip install -U pip
pip install wheel
pip install setuptools
pip install twine
pip install rst2pdf
pip install flit

# desactivar
deactivate

Guardar requisitos previos

Una vez completado el entorno virtual, conviene guardar el contexto. De esta forma, podrá restaurarlo rápidamente desde cero si es necesario.

Mientras está activado

pip freeze >requirements.txt

# Es  buena idea mantener dichos archivos bajo control de versiones.

Recuperar

Mientras está activado

pip install -r requirements.txt

Activación automatica

En muchos casos, queremos activar un proyecto al iniciar sesión. Podemos hacerlo automáticamente añadiendo algo como esto a .bashrc o .profile.

#
# Añadido para activación automatica del entorno virtual
#
if [ -f "${HOME}/.venv/devpi/bin/activate" ] ; then
    source ${HOME}/.venv/devpi/bin/activate
fi

O utilizando venvwrapper

#
# Añadido para activación automatica del entorno virtual
#
VENV_HOME=${HOME}/.venv
venv_wrapper=$(which venvwrapper.sh)
if [[ -n $venv_wrapper ]]; then
    source $venv_wrapper
    venv devpi
fi

python virtual environment

2025-05-24T18:23:52+00:00

Python Virtual Environment Setup

To have a good control over pre-requisite library modules for our software, we run in a python virtual environment.

That means that whenever you want to use or maintain the software you must first activate that virtual environment like this: (replace 'generic' with an existing project environment name)

source ${HOME}/venv_generic/bin/activate

Once active you can deactivate it like this:

deactivate

Installation

We install any non pip packages we rely on, if any, first. Then we install virtualenv. Then we create a virtual environment. Then we activate it.

Install virtualenv itself.

sudo apt update
sudo apt install virtualenv

Install a .rst to .pdf converter.

snap install rst2pdf

Create a virtual environment for the project

virtualenv --prompt venv_generic ${HOME}/venv_generic

Then activate the created environment

source ${HOME}/venv_generic/bin/activate

Now we install modules our project will need, first pip itself and the tools it needs.

python3 -m pip install -U pip
pip install wheel
pip install setuptools
pip install twine

We are using flit for building and installling our software.

pip install flit

Saving prerequisites

We keep our dependencies under version control, so each time we install more components we need to refresh our requirements file.

pip freeze >${HOME}/allrepos/generic/requirements.txt

Reinstalling

Later, when we are migrating to a different workstation, or for some reason need to restablish our project from scratch, we can install all our prerequisites in one shot instead of installing items one at a time like we did above.

pip install -r ${HOME}/allrepos/generic/requirements.txt

Automatic activation

In many cases we want to activate a project whenver we log in, so lets do that automatically.

Append this snippet to the end of our "${HOME}/.bashrc" file.

#
# Added for automatic vitual environment activation
#
if [ -f "${HOME}/venv_generic/bin/activate" ] ; then
    source ${HOME}/venv_generic/bin/activate
fi

Good ignoring keeps flit happy

Flit works nicely with git, but it is quite strict, so we must ignore all irrelevant files properly or flit will reject builds.

Below is a good start for a project ".gitignore" file:

*~
\#*\#
dist/*
pdf_docs/*
html_docs/*

Set Up A Wrapper For Venv Environments

2025-05-24T18:23:52+00:00

To maintain more than one python project at a time we leverage the the builtin python venv tool for creating and using virtual environments.

The venvwrapper package helps to keep the maintenance of multiple virtual environments easier. In short venv is enough, but venvwrapper makes it easier.

Installation

Install the venvwrapper package into your default system Python 3. That is, with no virtual environment active. This installs venvwrapper.sh so that it is available in PATH

sudo pip3 install --break-system-packages venvwrapper

Check that it is indeed available

which venvwrapper.sh

Set up automatic activation of venvwrapper

Add the code snippet below to one or both of these two files.

${HOME}/.bashrc
${HOME}/.profile

Code snippet:

VENV_HOME=${HOME}/.venv
venv_wrapper=$(which venvwrapper.sh)
if [[ -n ${venv_wrapper} ]]; then
    source ${venv_wrapper}
fi

Now Once you open a terminal window you have access to these commands

mkvenv devpi  # create and activate a new virtual environment
venv   devpi  # activate an existing virtual environment
rmvenv devpi  # remove one or more virtual environments
lsvenv        # list existing virtual environments