Changing a foreign key to the many-to-many field

This recipe is a practical example of how to change a many-to-one relation to a many-to-many relation, while preserving the already existing data. We will use both schema and data migrations in this situation.

Getting ready

Let's suppose that you have the Idea model, with a foreign key pointing to the Category model.

1. Let's define the Category model in the categories app, as follows:

# myproject/apps/categories/models.py
from django.db import models
from django.utils.translation import gettext_lazy as _

from myproject.apps.core.model_fields import MultilingualCharField


class Category(models.Model):
    title = MultilingualCharField(
        _("Title"),
        max_length=200,
    )

    class Meta:
        verbose_name = _("Category")
        verbose_name_plural = _("Categories")

    def __str__(self):
        return self.title

2. Let's define the Idea model in the ideas app, as follows:

# myproject/apps/ideas/models.py
from django.db import models
from django.conf import settings
from django.utils.translation import gettext_lazy as _

from myproject.apps.core.model_fields import (
    MultilingualCharField,
    MultilingualTextField,
)

class Idea(models.Model):
    title = MultilingualCharField(
        _("Title"),
        max_length=200,
    )
    content = MultilingualTextField(
        _("Content"),
    )
    category = models.ForeignKey(
        "categories.Category",
        verbose_name=_("Category"),
        blank=True,
        null=True,
        on_delete=models.SET_NULL,
        related_name="category_ideas",
    )

    class Meta:
        verbose_name = _("Idea")
        verbose_name_plural = _("Ideas")

    def __str__(self):
        return self.title

3. Create and execute initial migrations by using the following commands:

(env)$ python manage.py makemigrations categories
(env)$ python manage.py makemigrations ideas
(env)$ python manage.py migrate

How to do it...

The following steps will show you how to switch from a foreign key relation to a many-to-many relation, while preserving the already existing data:

1. Add a new many-to-many field, called categories, as follows:

# myproject/apps/ideas/models.py
from django.db import models
from django.conf import settings
from django.utils.translation import gettext_lazy as _

from myproject.apps.core.model_fields import (
    MultilingualCharField,
    MultilingualTextField,
)

class Idea(models.Model):
    title = MultilingualCharField(
        _("Title"),
        max_length=200,
    )
    content = MultilingualTextField(
        _("Content"),
    )
    category = models.ForeignKey(
        "categories.Category",
        verbose_name=_("Category"),
        blank=True,
        null=True,
        on_delete=models.SET_NULL,
        related_name="category_ideas",
    )
    categories = models.ManyToManyField(
        "categories.Category",
        verbose_name=_("Categories"),
        blank=True,
        related_name="ideas",
    )

    class Meta:
        verbose_name = _("Idea")
        verbose_name_plural = _("Ideas")

    def __str__(self):
        return self.title

2. Create and run a schema migration, in order to add the new relationship to the database, as shown in the following code snippet:

(env)$ python manage.py makemigrations ideas
(env)$ python manage.py migrate ideas

3. Create a data migration to copy the categories from the foreign key to the many-to-many field, as follows:

(env)$ python manage.py makemigrations --empty \
> --name=copy_categories ideas

4. Open the newly created migration file (0003_copy_categories.py), and define the forward migration instructions, as shown in the following code snippet:

# myproject/apps/ideas/migrations/0003_copy_categories.py
from django.db import migrations


def copy_categories(apps, schema_editor):
    Idea = apps.get_model("ideas", "Idea")
    for idea in Idea.objects.all():
        if idea.category:
            idea.categories.add(idea.category)


class Migration(migrations.Migration):

    dependencies = [
        ('ideas', '0002_idea_categories'),
    ]

    operations = [
        migrations.RunPython(copy_categories),
    ]

5. Run the new data migration, as follows:

(env)$ python manage.py migrate ideas

6. Delete the foreign key category field in the models.py file, leaving only the new categories many-to-many field, as follows:

# myproject/apps/ideas/models.py
from django.db import models
from django.conf import settings
from django.utils.translation import gettext_lazy as _

from myproject.apps.core.model_fields import (
    MultilingualCharField,
    MultilingualTextField,
)

class Idea(models.Model):
    title = MultilingualCharField(
        _("Title"),
        max_length=200,
    )
    content = MultilingualTextField(
        _("Content"),
    )

    categories = models.ManyToManyField(
        "categories.Category",
        verbose_name=_("Categories"),
        blank=True,
        related_name="ideas",
    )

    class Meta:
        verbose_name = _("Idea")
        verbose_name_plural = _("Ideas")

    def __str__(self):
        return self.title

7. Create and run a schema migration, in order to delete the Categories field from the database table, as follows:

(env)$ python manage.py makemigrations ideas
(env)$ python manage.py migrate ideas

How it works...

At first, we add a new many-to-many field to the Idea model, and a migration is generated to update the database accordingly. Then, we create a data migration that will copy the existing relations from the foreign key category to the new many-to-many categories. Lastly, we remove the foreign key field from the model, and update the database once more.

There's more...

Our data migration currently includes only the forward action, copying the foreign
key category as the first related item in the new categories relationship. Although we did not elaborate here, in a real-world scenario it would be best to include the reverse operation as well. This could be accomplished by copying the first related item back to the category foreign key. Unfortunately, any Idea object with multiple categories would lose extra data.

See also

• The Using migrations recipe
• The Handling multilingual fields recipe
• The Working with model translation tables recipe
• The Avoiding circular dependencies recipe