bug/regression in 4.2.4: Model typed through Generics has no attribute "objects"

[Andrioden]

Bug report

After upgrading from 4.2.3 -> 4.2.4 I am seeing different problems with typing related to [Model].objects in my codebase. Lazily i have only recreated one of the cases in minimalist example.

What's wrong

Case 1 - Model typed through Generics ++

from abc import ABC, abstractmethod
from typing import TypeVar, Type, Generic

from django.db.models import Model

M = TypeVar("M", bound=Model)


class BaseRepo(Generic[M], ABC):
    @classmethod
    @abstractmethod
    def _m(cls) -> Type[M]:
        pass

    @classmethod
    def count_all(cls) -> int:
        return cls._m().objects.count()

->

game\repos\regr.py:17: error: Returning Any from function declared to return "int"  [no-any-return]
game\repos\regr.py:17: error: "type[M]" has no attribute "objects"  [attr-defined]

Case 2 - ManyToMany objects ref

class CrittDuel(models.Model):
    log = models.ManyToManyField(LogEvent)

CrittDuelLog = CrittDuel.log.through
CrittDuelLog.objects.bulk_create(bulk_create_log_m2m) # <---- HERE

->

game\battle\services.py:165: error: "type[Model]" has no attribute "objects"  [attr-defined]

How is that should be

No mypy errors. Works with 4.2.3.

System information

• OS:
• python version: 3.11.3
• django version: 4.2.4
• mypy version: 1.5.1
• django-stubs version: 4.2.4
• django-stubs-ext version: 4.2.2

[flaeppe]

Duplicate of #1684

[Andrioden]

I actually read that documentation before posting this and while it may not be your fault or documentation, this is very confusing.

Why would this be ok

from django.db.models import Model
class CoolClass(Model):
    pass

CoolClass.objects.count()

and not this

from django.db.models import Model
M = TypeVar("M", bound=Model)
Type[M].objects.count()  # Not exactly like this, but you get my point

You are using exactly the same type information Model, how can mypy (you?) trust that CoolClass.objects.count() works but not Type[M].objects.count()? It does not seem consistent.

Should CoolClass.objects.count() also raise an error then? And doesnt that mean everyone should be writing CoolClass._base_manager.count()? This cant be django's design choice.

[Andrioden]

Maybe i need to read up on the god forsaken TypeVar definition again -.-

[flaeppe]

from django.db.models import Model
class CoolClass(Model):
    pass

CoolClass.objects.count()

CoolClass is a direct subclass of models.Model, and what django does then is that adds an implicit default manager, unless the model has declared one by itself. CoolClass hasn't, as such .objects is added. It's probably documented somewhere as well, but you can see it in action here: https://github.com/django/django/blob/cafe7266ee69f7e017ddbc0d440084ace559b04b/django/db/models/base.py#L411-L419

The important difference compared to

T = TypeVar("T", bound=models.Model)

Is that we set an upper bound to the parent class of all models here. I can very well have done what you see below, to my models.Model. Which is indeed a models.Model, but doesn't have a manager .objects because I've decided to create my very own default manager.

class MyCoolModel(models.Model):
    not_objects = models.Manager()

print(type(MyCoolModel.not_objects))
try:
    MyCoolModel.objects
except AttributeError:
    print("Oh no")

---

The key thing to take away here is that when we're talking about the base class models.Model it doesn't declare any attribute .objects (you'll find the guard here). objects is dynamically added to a subclass of models.Model depending on a couple of preconditions.

[Andrioden]

The key thing to take away here is that when we're talking about the base class models.Model it doesn't declare any attribute .objects (you'll find the guard here). objects is dynamically added to a subclass of models.Model depending on a couple of preconditions.

Thank you for this explanation, its kinda bad design by django, but at least i understand it, and django-stubs certainly are behaving correct. Thank you again.

[Alexerson]

I can open a new issue if it’s more relevant, but I feel the 2nd use case was a bit overlooked, but is a real issue:

class CrittDuel(models.Model):
    log = models.ManyToManyField(LogEvent)

CrittDuelLog = CrittDuel.log.through
CrittDuelLog.objects.bulk_create(bulk_create_log_m2m) # <---- HERE

The last line of the paragraph in Django’s doc about through (https://docs.djangoproject.com/en/4.2/ref/models/fields/#django.db.models.ManyToManyField.through) clearly states that objects should be defined in that case.

Am I missing something?

[flaeppe]

clearly states that objects should be defined in that case.

I don't see where it clearly states so? I see this though

This class can be used to query associated records for a given model instance like a normal model:

So, that means it behaves just regularly regarding default/first declared managers I think?

Perhaps there's an improvement if we sort out that there's no explicit through= declared, which means that you can directly assume objects exists. But as soon as through= is declared you'd have to inspect that model to know if there's an objects or not.

Anyway, we still have #1549 to fix first. So if through= can be resolved appropriately to a model, the rest should work too. Much like has been done for e.g. ForeignKey. And that should be done first, IMO

[Alexerson]

I read:

This class can be used to query associated records for a given model instance like a normal model:

Model.m2mfield.through.objects.all()

So it assumes objects is defined, no?

But yeah, fixing #1549 is probably the proper plan.

[Alexerson]

Ok, I re-read your answer. I actually agree with what your saying (re: implicit vs explicit through declaration).
We’re aligned then. I’ll try to find some time in the next weeks to look again at #1549 and see what I can come up with.