diff --git a/doc/admin/config.rst b/doc/admin/config.rst index d3418e7c00..e8a8d4aca3 100644 --- a/doc/admin/config.rst +++ b/doc/admin/config.rst @@ -141,7 +141,7 @@ Database settings Example:: [database] - backend=mysql + backend=postgresql name=pretix user=pretix password=abcd @@ -149,7 +149,7 @@ Example:: port=3306 ``backend`` - One of ``mysql``, ``sqlite3``, ``oracle`` and ``postgresql``. + One of ``mysql`` (deprecated), ``sqlite3`` and ``postgresql``. Default: ``sqlite3``. If you use MySQL, be sure to create your database using @@ -163,7 +163,7 @@ Example:: Connection details for the database connection. Empty by default. ``galera`` - Indicates if the database backend is a MySQL/MariaDB Galera cluster and + (Deprecated) Indicates if the database backend is a MySQL/MariaDB Galera cluster and turns on some optimizations/special case handlers. Default: ``False`` .. _`config-replica`: diff --git a/doc/admin/index.rst b/doc/admin/index.rst index f60d61b703..b0fe072f54 100644 --- a/doc/admin/index.rst +++ b/doc/admin/index.rst @@ -14,4 +14,5 @@ This documentation is for everyone who wants to install pretix on a server. maintainance scaling errors + mysql2postgres indexes diff --git a/doc/admin/installation/docker_smallscale.rst b/doc/admin/installation/docker_smallscale.rst index 6c5f2a85a5..c172780cab 100644 --- a/doc/admin/installation/docker_smallscale.rst +++ b/doc/admin/installation/docker_smallscale.rst @@ -14,7 +14,7 @@ This has some trade-offs in terms of performance and isolation but allows a rath get it right. If you're not feeling comfortable managing a Linux server, check out our hosting and service offers at `pretix.eu`_. -We tested this guide on the Linux distribution **Debian 8.0** but it should work very similar on other +We tested this guide on the Linux distribution **Debian 11.0** but it should work very similar on other modern distributions, especially on all systemd-based ones. Requirements @@ -26,7 +26,7 @@ installation guides): * `Docker`_ * A SMTP server to send out mails, e.g. `Postfix`_ on your machine or some third-party server you have credentials for * A HTTP reverse proxy, e.g. `nginx`_ or Apache to allow HTTPS connections -* A `PostgreSQL`_ 9.6+, `MySQL`_ 5.7+, or MariaDB 10.2.7+ database server +* A `PostgreSQL`_ 9.6+ database server * A `redis`_ server We also recommend that you use a firewall, although this is not a pretix-specific recommendation. If you're new to @@ -58,9 +58,6 @@ directory writable to the user that runs pretix inside the docker container:: Database -------- -.. warning:: **Please use PostgreSQL for all new installations**. If you need to go for MySQL, make sure you run - **MySQL 5.7 or newer** or **MariaDB 10.2.7 or newer**. - Next, we need a database and a database user. We can create these with any kind of database managing tool or directly on our database's shell. Please make sure that UTF8 is used as encoding for the best compatibility. You can check this with the following command:: @@ -86,13 +83,6 @@ Restart PostgreSQL after you changed these files:: If you have a firewall running, you should also make sure that port 5432 is reachable from the ``172.17.0.1/16`` subnet. -For MySQL, you can either also use network-based connections or mount the ``/var/run/mysqld/mysqld.sock`` socket into the docker container. -When using MySQL, make sure you set the character set of the database to ``utf8mb4``, e.g. like this:: - - mysql > CREATE DATABASE pretix DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci; - -You will also need to make sure that ``sql_mode`` in your ``my.cnf`` file does **not** include ``ONLY_FULL_GROUP_BY``. - Redis ----- @@ -152,15 +142,13 @@ Fill the configuration file ``/etc/pretix/pretix.cfg`` with the following conten trust_x_forwarded_proto=on [database] - ; Replace postgresql with mysql for MySQL backend=postgresql name=pretix user=pretix ; Replace with the password you chose above password=********* ; In most docker setups, 172.17.0.1 is the address of the docker host. Adjust - ; this to wherever your database is running, e.g. the name of a linked container - ; or of a mounted MySQL socket. + ; this to wherever your database is running, e.g. the name of a linked container. host=172.17.0.1 [mail] @@ -212,8 +200,6 @@ named ``/etc/systemd/system/pretix.service`` with the following content:: [Install] WantedBy=multi-user.target -When using MySQL and socket mounting, you'll need the additional flag ``-v /var/run/mysqld:/var/run/mysqld`` in the command. - You can now run the following commands to enable and start the service:: @@ -339,7 +325,6 @@ workers, e.g. ``docker run … taskworker -Q notifications --concurrency 32``. .. _nginx: https://botleg.com/stories/https-with-lets-encrypt-and-nginx/ .. _Let's Encrypt: https://letsencrypt.org/ .. _pretix.eu: https://pretix.eu/ -.. _MySQL: https://dev.mysql.com/doc/refman/5.7/en/linux-installation-apt-repo.html .. _PostgreSQL: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-postgresql-on-ubuntu-20-04 .. _redis: https://blog.programster.org/debian-8-install-redis-server/ .. _ufw: https://en.wikipedia.org/wiki/Uncomplicated_Firewall diff --git a/doc/admin/installation/manual_smallscale.rst b/doc/admin/installation/manual_smallscale.rst index 498b35b82a..8bb09b6c53 100644 --- a/doc/admin/installation/manual_smallscale.rst +++ b/doc/admin/installation/manual_smallscale.rst @@ -23,7 +23,7 @@ installation guides): * A SMTP server to send out mails, e.g. `Postfix`_ on your machine or some third-party server you have credentials for * A HTTP reverse proxy, e.g. `nginx`_ or Apache to allow HTTPS connections -* A `PostgreSQL`_ 9.6+, `MySQL`_ 5.7+, or MariaDB 10.2.7+ database server +* A `PostgreSQL`_ 9.6+ database server * A `redis`_ server * A `nodejs`_ installation @@ -47,9 +47,6 @@ In this guide, all code lines prepended with a ``#`` symbol are commands that yo Database -------- -.. warning:: **Please use PostgreSQL for all new installations**. If you need to go for MySQL, make sure you run - **MySQL 5.7 or newer** or **MariaDB 10.2.7 or newer**. - Having the database server installed, we still need a database and a database user. We can create these with any kind of database managing tool or directly on our database's shell. Please make sure that UTF8 is used as encoding for the best compatibility. You can check this with the following command:: @@ -61,12 +58,6 @@ For PostgreSQL database creation, we would do:: # sudo -u postgres createuser pretix # sudo -u postgres createdb -O pretix pretix -When using MySQL, make sure you set the character set of the database to ``utf8mb4``, e.g. like this:: - - mysql > CREATE DATABASE pretix DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci; - -You will also need to make sure that ``sql_mode`` in your ``my.cnf`` file does **not** include ``ONLY_FULL_GROUP_BY``. - Package dependencies -------------------- @@ -74,7 +65,7 @@ To build and run pretix, you will need the following debian packages:: # apt-get install git build-essential python-dev python3-venv python3 python3-pip \ python3-dev libxml2-dev libxslt1-dev libffi-dev zlib1g-dev libssl-dev \ - gettext libpq-dev libmariadb-dev libjpeg-dev libopenjp2-7-dev + gettext libpq-dev libjpeg-dev libopenjp2-7-dev Config file ----------- @@ -97,16 +88,12 @@ Fill the configuration file ``/etc/pretix/pretix.cfg`` with the following conten trust_x_forwarded_proto=on [database] - ; For MySQL, replace with "mysql" backend=postgresql name=pretix user=pretix - ; For MySQL, enter the user password. For PostgreSQL on the same host, - ; we don't need one because we can use peer authentification if our - ; PostgreSQL user matches our unix user. + ; For PostgreSQL on the same host, we don't need a password because we can use + ; peer authentication if our PostgreSQL user matches our unix user. password= - ; For MySQL, use local socket, e.g. /var/run/mysqld/mysqld.sock - ; For a remote host, supply an IP address ; For local postgres authentication, you can leave it empty host= @@ -140,10 +127,6 @@ We now install pretix, its direct dependencies and gunicorn:: (venv)$ pip3 install pretix gunicorn -If you're running MySQL, also install the client library:: - - (venv)$ pip3 install mysqlclient - Note that you need Python 3.7 or newer. You can find out your Python version using ``python -V``. We also need to create a data directory:: @@ -344,7 +327,6 @@ Then, proceed like after any plugin installation:: .. _nginx: https://botleg.com/stories/https-with-lets-encrypt-and-nginx/ .. _Let's Encrypt: https://letsencrypt.org/ .. _pretix.eu: https://pretix.eu/ -.. _MySQL: https://dev.mysql.com/doc/refman/5.7/en/linux-installation-apt-repo.html .. _PostgreSQL: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-postgresql-on-ubuntu-20-04 .. _redis: https://blog.programster.org/debian-8-install-redis-server/ .. _ufw: https://en.wikipedia.org/wiki/Uncomplicated_Firewall diff --git a/doc/admin/maintainance.rst b/doc/admin/maintainance.rst index b692f564f2..30b230e8c5 100644 --- a/doc/admin/maintainance.rst +++ b/doc/admin/maintainance.rst @@ -17,11 +17,11 @@ Backups There are essentially two things which you should create backups of: Database - Your SQL database (MySQL or PostgreSQL). This is critical and you should **absolutely - always create automatic backups of your database**. There are tons of tutorials on the - internet on how to do this, and the exact process depends on the choice of your database. - For MySQL, see ``mysqldump`` and for PostgreSQL, see the ``pg_dump`` tool. You probably - want to create a cronjob that does the backups for you on a regular schedule. + Your SQL database. This is critical and you should **absolutely always create automatic + backups of your database**. There are tons of tutorials on the internet on how to do this, + and the exact process depends on the choice of your database. For PostgreSQL, see the + ``pg_dump`` tool. You probably want to create a cronjob that does the backups for you on a + regular schedule. Data directory The data directory of your pretix configuration might contain some things that you should diff --git a/doc/admin/mysql2postgres.rst b/doc/admin/mysql2postgres.rst new file mode 100644 index 0000000000..29a46d7334 --- /dev/null +++ b/doc/admin/mysql2postgres.rst @@ -0,0 +1,148 @@ +.. highlight:: none + +Migrating from MySQL/MariaDB to PostgreSQL +========================================== + +Our recommended database for all production installations is PostgreSQL. Support for MySQL/MariaDB will be removed in +pretix 5.0. + +In order to follow this guide, your pretix installation needs to be a version that fully supports MySQL/MariaDB. If you +already upgraded to pretix 5.0, downgrade back to the last 4.x release using ``pip``. + +.. note:: We have tested this guide carefully, but we can't assume any liability for its correctness. The data loss + risk should be low as long as pretix is not running while you do the migration. If you are a pretix Enterprise + customer, feel free to reach out in advance if you want us to support you along the way. + +Update database schema +---------------------- + +Before you start, make sure your database schema is up to date:: + + # sudo -u pretix -s + $ source /var/pretix/venv/bin/activate + (venv)$ python -m pretix migrate + +Install PostgreSQL +------------------ + +Now, install and set up a PostgreSQL server. For a local installation on Debian or Ubuntu, use:: + + # apt install postgresql + +Having the database server installed, we still need a database and a database user. We can create these with any kind +of database managing tool or directly on our database's shell. Please make sure that UTF8 is used as encoding for the +best compatibility. You can check this with the following command:: + + # sudo -u postgres psql -c 'SHOW SERVER_ENCODING' + +Without Docker +"""""""""""""" + +For our standard manual installation, create the database and user like this:: + + # sudo -u postgres createuser pretix + # sudo -u postgres createdb -O pretix pretix + +With Docker +""""""""""" + +For our standard docker installation, create the database and user like this:: + + # sudo -u postgres createuser -P pretix + # sudo -u postgres createdb -O pretix pretix + +Make sure that your database listens on the network. If PostgreSQL on the same same host as docker, but not inside a docker container, we recommend that you just listen on the Docker interface by changing the following line in ``/etc/postgresql//main/postgresql.conf``:: + + listen_addresses = 'localhost,172.17.0.1' + +You also need to add a new line to ``/etc/postgresql//main/pg_hba.conf`` to allow network connections to this user and database:: + + host pretix pretix 172.17.0.1/16 md5 + +Restart PostgreSQL after you changed these files:: + + # systemctl restart postgresql + +If you have a firewall running, you should also make sure that port 5432 is reachable from the ``172.17.0.1/16`` subnet. + +Of course, instead of all this you can also run a PostgreSQL docker container and link it to the pretix container. + +Stop pretix +----------- + +To prevent any more changes to your data, stop pretix from running:: + + # systemctl stop pretix-web pretix-worker + +Change configuration +-------------------- + +Change the database configuration in your ``/etc/pretix/pretix.cfg`` file:: + + [database] + backend=postgresql + name=pretix + user=pretix + password= ; only required for docker or remote database, can be kept empty for local auth + host= ; set to 172.17.0.1 in docker setup, keep empty for local auth + + +Create database schema +----------------------- + +To create the schema in your new PostgreSQL database, use the following commands:: + + # sudo -u pretix -s + $ source /var/pretix/venv/bin/activate + (venv)$ python -m pretix migrate + + +Migrate your data +----------------- + +Install ``pgloader``:: + + # apt install pgloader + +Create a new file ``/tmp/pretix.load``, replacing the MySQL and PostgreSQL connection strings with the correct user names, passwords, and/or database names:: + + LOAD DATABASE + FROM mysql://pretix:password@localhost/pretix -- replace with mysql://username:password@hostname/dbname + INTO postgresql:///pretix -- replace with dbname + + WITH data only, include no drop, truncate, disable triggers, + create no indexes, drop indexes, reset sequences + + ALTER SCHEMA 'pretix' RENAME TO 'public' -- replace pretix with the name of the MySQL database + + ALTER TABLE NAMES MATCHING ~/.*/ + SET SCHEMA 'public' + + SET timezone TO '+00:00' + + SET PostgreSQL PARAMETERS + maintenance_work_mem to '128MB', + work_mem to '12MB'; + +Then, run:: + + # sudo -u postgres pgloader /tmp/pretix.load + +The output should end with a table summarizing the results for every table. You can ignore warnings about type casts +and missing constraints. + +Afterwards, delete the file again:: + + # rm -rf /tmp/pretix.load + +Start pretix +------------ + +Now, restart pretix. Maybe stop your MySQL server as a verification step that you are no longer using it:: + + # systemctl stop mariadb + # systemctl start pretix-web pretix-worker + +And you're done! After you've verified everything has been copied correctly, you can delete the old MySQL database. + +.. note:: Don't forget to update your backup process to back up your PostgreSQL database instead of your MySQL database now. diff --git a/doc/admin/scaling.rst b/doc/admin/scaling.rst index 17d761fb1f..f54a56c6de 100644 --- a/doc/admin/scaling.rst +++ b/doc/admin/scaling.rst @@ -42,7 +42,7 @@ A pretix installation usually consists of the following components which run per * ``pretix-worker`` is a Celery-based application that processes tasks that should be run asynchronously outside of the web application process. -* A **SQL database** keeps all the important data and processes the actual transactions. We recommend using PostgreSQL, but MySQL/MariaDB works as well. +* A **PostgreSQL database** keeps all the important data and processes the actual transactions. * A **web server** that terminates TLS and HTTP connections and forwards them to ``pretix-web``. In some cases, e.g. when serving static files, the web servers might return a response directly. We recommend using ``nginx``. diff --git a/src/pretix/control/templates/pretixcontrol/base.html b/src/pretix/control/templates/pretixcontrol/base.html index 95ea282858..9454ff66cd 100644 --- a/src/pretix/control/templates/pretixcontrol/base.html +++ b/src/pretix/control/templates/pretixcontrol/base.html @@ -423,6 +423,17 @@ {% endif %} + {% if "mysql" in settings.DATABASES.default.ENGINE and not request.organizer %} +
+ {% blocktrans trimmed %} + You are using MySQL or MariaDB as your database backend for pretix. + Starting in pretix 5.0, these will no longer be supported and you will need to migrate to PostgreSQL. + Please see the pretix administrator documentation for a migration guide, and the pretix 4.16 + release notes for more information. + {% endblocktrans %} +
+ {% endif %} + {% if debug_warning %}
{% trans "pretix is running in debug mode. For security reasons, please never run debug mode on a production instance." %}