Full support for nested pydantic, sqlmodel and ormar models #1637

gazorby · 2022-02-10T20:48:47Z

This PR add the following features:

Full support for deriving nested pydantic models (including when using List, Optional, Union and ForwardRef)
Deriving ormar models with relationships (ForeignKey, ManyToMany, and reverse relations)
Deriving SQLModel models with Relationship fields
Strawberry types declarations don't have to follow model declaration order (eg: children can be defined before parents)
Add a new exclude param to the strawberry.experimental.pydantic.type decorator, allowing to include all fields while excluding some

Description

Pydantic

Nested pydantic models should works in most situations, as long as it's GraphQL typed.

Example :

class User(pydantic.BaseModel):
    name: str
    hobby: Optional[List["Hobby"]]

class Hobby(pydantic.BaseModel):
    name: str

@strawberry.experimental.pydantic.type(User, all_fields=True)
class UserType:
    pass

@strawberry.experimental.pydantic.type(Hobby, all_fields=True)
class HobbyType:
    pass

The order in which strawberry types are defined doesn't matter (doesn't have to follow pydantic models declaration order):

class Hobby(pydantic.BaseModel):
    name: str

class User(pydantic.BaseModel):
    name: str
    hobby: Hobby

@strawberry.experimental.pydantic.type(User, all_fields=True)
class UserType:
    pass

@strawberry.experimental.pydantic.type(Hobby, all_fields=True)
class HobbyType:
    pass

Ormar

Omar is an orm that uses pydantic as its base: all ormar models are pydantic models.

ForeignKey, ManyToMany and reverse relations (related fields in Django) are supported.

When using the pydantic decorator to generate a strawberry type, ormar fields will be mapped as you would expect:

class Hobby(ormar.Model):
    name: str    

class User(ormar.Model):
    name: str = ormar.String(max_length=255)
    hobby: Hobby = ormar.ForeignKey(Hobby, nullable=False)

@strawberry.experimental.pydantic.type(Hobby, all_fields=True)
class HobbyType:
    pass

@strawberry.experimental.pydantic.type(User, all_fields=True)
class UserType:
    pass

Will gives the following GraphQL schema:

type HobbyType {
  name: String!
}

type UserType {
  name: String!
  hobby: HobbyType!
}

When using all_fields=True it also includes the reverse relation users that ormar automatically created in the Hobby model:

@strawberry.experimental.pydantic.type(Hobby, all_fields=True)
class HobbyType:
    pass

type HobbyType {
  name: String!
  users: [UserType]
}

type UserType {
  name: String!
  hobby: HobbyType!
}

SQLModel

SQLModel is another pydantic-based orm, that uses SQLAlchemy to define models. All relations are defined using the Relationship field:

class Hobby(SQLModel):
    name: str
    users: List["User"] = Relationship(back_populates="hobby")

class User(SQLModel):
    name: str = Field()
    hobby: Hobby = Relationship(back_populates="users")

@strawberry.experimental.pydantic.type(Hobby, all_fields=True)
class HobbyType:
    pass

@strawberry.experimental.pydantic.type(User, all_fields=True)
class UserType:
    pass

Will translate in the following schema:

type HobbyType {
  name: String!
  users: [UserType!]!
}

type UserType {
  name: String!
  hobby: HobbyType!
}

Types of Changes

Issues Fixed or Closed by This PR

Integrate with SQLModel #1183

Checklist

My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have read the CONTRIBUTING document.
I have added tests to cover my changes.
I have tested the changes and verified that they work and don't break anything (as well as I can manage).

botberry · 2022-02-10T20:51:21Z

Thanks for adding the RELEASE.md file!

Here's a preview of the changelog:

Add Full support for deriving nested pydantic models (including when using List, Optional, Union and ForwardRef)
Support for deriving ormar models with relationships (ForeignKey, ManyToMany, and reverse relations)
Support for deriving SQLModel models with Relationship fields
Strawberry type declarations don't have to follow model declarations order (eg: childs can be defined before parents)
Add a new exclude param to the strawberry.experimental.pydantic.type decorator, allowing to include all fields while excluding some

Pydantic

GraphQL container types (List, Optional and Union) and ForwardRef are supported:

class User(pydantic.BaseModel):
    name: str
    hobby: Optional[List["Hobby"]]

class Hobby(pydantic.BaseModel):
    name: str

@strawberry.experimental.pydantic.type(User, all_fields=True)
class UserType:
    pass

@strawberry.experimental.pydantic.type(Hobby, all_fields=True)
class HobbyType:
    pass

Ormar

ForeignKey, ManyToMany and reverse relations are supported:

class Hobby(ormar.Model):
    name: str

class User(ormar.Model):
    name: str = ormar.String(max_length=255)
    hobby: Hobby = ormar.ForeignKey(Hobby, nullable=False)

@strawberry.experimental.pydantic.type(Hobby, all_fields=True)
class HobbyType:
    pass

@strawberry.experimental.pydantic.type(User, all_fields=True)
class UserType:
    pass

type HobbyType {
  name: String!
  users: [UserType]
}

type UserType {
  name: String!
  hobby: HobbyType!
}

SLQModel

SQLModel is another pydantic-based orm, that uses SQLAlchemy to define models. All relations are defined using the Relationship field:

class Hobby(SQLModel, table=True):
    name: str
    users: List["User"] = Relationship(back_populates="hobby")

class User(SQLModel, table=True):
    name: str = Field()
    hobby: Hobby = Relationship(back_populates="users")

@strawberry.experimental.pydantic.type(Hobby, all_fields=True)
class HobbyType:
    pass

@strawberry.experimental.pydantic.type(User, all_fields=True)
class UserType:
    pass

type HobbyType {
  name: String!
  users: [UserType!]!
}

type UserType {
  name: String!
  hobby: HobbyType!
}

Here's the preview release card for twitter:

Here's the tweet text:

🆕 Release (next) is out! Thanks to @gazorby for this great new feature!
Strawberry types can now be generated from SQLModel / ormar models!

Get it here 👉 https://github.com/strawberry-graphql/strawberry/releases/tag/(next)

patrick91 · 2022-02-10T21:29:52Z

Hi @gazorby! Thanks for this PR, I'll try to take a look in the weekend <3

codecov · 2022-02-10T21:32:06Z

Codecov Report

Merging #1637 (6ac1045) into main (f2cc503) will decrease coverage by 0.02%.
The diff coverage is 99.26%.

❗ Current head 6ac1045 differs from pull request most recent head 65da6f2. Consider uploading reports for the commit 65da6f2 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1637      +/-   ##
==========================================
- Coverage   98.14%   98.12%   -0.02%     
==========================================
  Files         129      131       +2     
  Lines        4528     4649     +121     
  Branches      779      799      +20     
==========================================
+ Hits         4444     4562     +118     
- Misses         43       45       +2     
- Partials       41       42       +1

gazorby · 2022-02-12T13:56:13Z

Hi @patrick91!
Will update the doc accordingly once API changes have been finalized

mypy_tests.ini

strawberry/experimental/pydantic/nested/base.py

alexhafner · 2022-02-17T12:14:40Z

pyproject.toml

+sqlmodel = {version = "^0.0.6", optional = true}
+ormar = {version = "^0.10.24", optional = true}


The inclusion of sqlmodel and ormar had an impact on sqlalchemy which I had locked to 1.4.31 in my test project, ie it forced a downgrade like so:

Because strawberry-graphql (0.96.0) depends on ormar (>=0.10.24,<0.11.0) and no versions of ormar match >0.10.24,<0.11.0, strawberry-graphql (0.96.0) requires ormar (0.10.24). And because ormar (0.10.24) depends on SQLAlchemy (>=1.3.18,<=1.4.29), strawberry-graphql (0.96.0) requires SQLAlchemy (>=1.3.18,<=1.4.29). And because sqlalchemy (1.4.31) depends on sqlalchemy (1.4.31) and no versions of sqlalchemy match >1.4.31,<2.0.0, strawberry-graphql (0.96.0) is incompatible with SQLAlchemy (>=1.4.31,<2.0.0). So, because main-api-server depends on both SQLAlchemy (^1.4.31) and strawberry-graphql (0.96.0), version solving failed.

after changing the pin on 1.4.31 the changes were as follows

Updating sqlalchemy (1.4.31 -> 1.4.29) • Installing databases (0.5.4) • Installing ormar (0.10.24) • Installing sqlmodel (0.0.6) • Updating strawberry-graphql (0.97.0 -> 0.96.0 /test/strawberry/dist/strawberry_graphql-0.96.0-py3-none-any.whl)

They're currently marked as optional; adding them to [tool.poetry.extras] prevents them to be installed by default

Thanks, b95dd95 has fixed that

alexhafner · 2022-02-17T23:35:55Z

strawberry/experimental/pydantic/object_type.py

    if hasattr(type_, "__args__"):
        replaced_type = type_.copy_with(
-            tuple(replace_pydantic_types(t, is_input) for t in type_.__args__)
+            tuple(
+                replace_pydantic_types(t, is_input, model, name) for t in type_.__args__
+            )
        )


Hi @gazorby

Many thanks for your work on this :-)

I'm testing the PR against a set of nested Pydantic Models. In this section, I'm running into an issue with pydantic fields using a list like orders: Optional[list[Order]] and using a dict like name_dir: dict[str, Any] where it then throws AttributeError: type object 'dict' has no attribute 'copy_with' or AttributeError: type object 'list' has no attribute 'copy_with'

Hi @alexhafner

Thanks for your reports, I think we don't support builtin types (like list) yet, only those from the typing module. I tested with the following nested structure and it works:

class HobbyModel(pydantic.BaseModel): name: str class UserModel(pydantic.BaseModel): age: int hobbies: Optional[List[HobbyModel]] @strawberry.experimental.pydantic.type(HobbyModel, fields=["name"]) class Hobby: pass @strawberry.experimental.pydantic.type(UserModel, fields=["age", "hobbies"]) class User: pass

Thanks for that @gazorby - I have changed the underlying pydantic models using list[] to List[] and it works fine. dict[...] or Dict[...] doesn't seem to be supported at all.

some of the checks in if issubclass(type_, BaseModel): fail with issubclass() arg 1 must be a class. For example if you try to use Dict[str, str] you get TypeError: Foo fields cannot be resolved. Unexpected type 'typing.Dict[str, str]'. With Dict[str, Any] you end up with TypeError: issubclass() arg 1 must be a class. Once you add an additional class check like if inspect.isclass(type_) and issubclass(type_, BaseModel): then the expected error is returned TypeError: Foo fields cannot be resolved. Unexpected type 'typing.Dict[str, typing.Any]'

Does strawberry currently support typing containers, e.g. typing.Set? I see errors like

TypeError: Unexpected type 'typing.Set[pydantic.networks.EmailStr]'

GraphQL doesn't have a Set container type, only List. You'll have to convert it to list before you instantiate the strawberry ObjectType

Oh, gotcha, thanks for the pointer and apologies for hijacking the PR here 🙇

For anyone landing on this page, here are the docs and my code to solve this:

@staticmethod def from_pydantic( instance: MyPydanticModel, extra: Optional[Dict[str, Any]] = None, ) -> 'MyModel': data = instance.dict() data['emails'] = list(data.get('emails', [])) return MyModel(**data) def to_pydantic(self) -> MyPydanticModel: data = dataclasses.asdict(self) data['emails'] = {EmailStr(email) for email in data.get('emails', [])} return MyPydanticModel(**data)

alexhafner · 2022-02-18T00:28:01Z

strawberry/experimental/pydantic/object_type.py

+def replace_pydantic_types(
+    type_: Any, is_input: bool, model: Type[BaseModel], name: str
+):


I did also run in the Enum issue throwing `TypeError: StrawberryModel fields cannot be resolved. _enum_definition` raised in #1598. The solution suggested there, with the addition of a type check before the issubclass check, solved the issue for me, along the lines of

import inspect
from enum import Enum
# ...
if inspect.isclass(type_) and issubclass(type_, Enum):
strawberry.enum(type_)

patrick91 · 2022-03-05T05:07:12Z

@gazorby thank you so much for doing this! is there any chance we can split this PR into multiple ones? it would make it easier to get things merged 😊

alexhafner · 2022-05-07T15:02:42Z

@gazorby @patrick91 I wonder how we could move this forward / split the PR if required. I've used @gazorby 's fork in a POC for a while (the Pydantic features, not ormar or SQLModel) and I feel features like the nested models delivered here are essential for most non-trivial Strawberry implementations using Pydantic.

With the (thankfully!) frequent releases of Strawberry though this is not sustainable for long as we want to uptake the latest features and releases, and some pydantic related code has also been moving again. Would be fantastic to be able to merge the functionality soon.

patrick91 · 2022-05-07T15:07:39Z

Hi @alexhafner, thanks for the message, that's great to know! I might need to do some pydantic+strawberry stuff for work so I should be able to take a look at this PR :)

Are you on discord by the way? I might want to ask you some questions regarding this PR 😊

alexhafner · 2022-05-07T15:13:20Z

@patrick91 sure thing, I'll message you there

gazorby · 2022-05-07T21:45:36Z

Hi @alexhafner @patrick91

Sorry for being silent here, was busy on other projects.
I think nested model generation is important for any advanced use-case making use of pydantic or any other model-like abstraction. I recently came through the great expedock/strawberry-sqlalchemy-mapper library that help generating strawberry types from sqlalchemy models, and forked it to add input types generation and full relay support (including pagination) to their already exisiting relay implementation.

I realized that doing the same in strawberry if this PR was merged would be quite cumbersome as it lacks some way of customizing what's happening during nested generation (eg: define a resolver that would accept a page input and return a connection). Of course we don't want to bloat more this PR by adding relay support, but maybe some bindings (decorator param?) to let strawberry users customize how nested types would be generated.

patrick91 · 2022-05-07T22:26:57Z

Hi @alexhafner @patrick91

Sorry for being silent here, was busy on other projects.

no worries at all!

I think nested model generation is important for any advanced use-case making use of pydantic or any other model-like abstraction.

Yup, I agree 😌

I recently came throught the great expedock/strawberry-sqlalchemy-mapper library that help generating strawberry types from sqlalchemy models, and forked it to add input types generation and full relay support (including pagination) to their already exisiting relay implementation.

Did you make a PR? would be interesting to see how you implemented it

I realized that doing the same in strawbrry if this PR was merged would be quite cumbersome as it lacks some way of customizing what's happening during nested generation (eg: define a resolver that would accept a page input and return a connection). Of course we don't to bloat more this PR by adding relay support, but maybe some bindings (decorator param?) to let strawberry users customize how nested types would be generated.

Would you be able to write a summary for this?

Also what do you think of splitting this PR in multiple pieces?

ElisaKonga · 2022-07-26T10:08:48Z

Hi! Is this feature development still in active development?

khairm · 2022-12-11T00:26:10Z

@gazorby a lot of great work here. Worth breaking down what can be merged straight away into a separate pr?

pranav-kunapuli · 2023-01-19T16:24:00Z

@gazorby These are fantastic changes! Any chance we can get them merged soon?

cat-turner · 2023-01-24T09:27:41Z

hi, I need exclude pls this PR looks great

gazorby · 2023-01-30T14:53:36Z

Unfortunately I have no more interest in deriving strawberry types from sqlmodel/ormar models and pydantic first class support is tracked in #2181.
For anyone willing to iterate from there, initial work is at https://github.com/gazorby/strawberry/tree/ref/simpler-orm-implementation

gazorby changed the title ~~Full support for nested pydantic, sqlmodel and ormar~~ Full support for nested pydantic, sqlmodel and ormar models Feb 10, 2022

botberry added bot:has-release-file labels Feb 10, 2022

patrick91 self-requested a review February 10, 2022 21:29

gazorby force-pushed the main branch from 5d4d304 to 02da088 Compare February 12, 2022 13:50

BryceBeagle reviewed Feb 13, 2022

View reviewed changes

alexhafner reviewed Feb 17, 2022

View reviewed changes

gazorby force-pushed the main branch from dc27ee5 to 85f1098 Compare February 17, 2022 20:32

alexhafner reviewed Feb 17, 2022

View reviewed changes

alexhafner reviewed Feb 18, 2022

View reviewed changes

bugzpodder mentioned this pull request Feb 24, 2022

all_fields in pydantic definition should allow overrides #1679

Open

botberry removed the bot:has-release-file label Jan 11, 2023

botberry added the bot:has-release-file label Jan 30, 2023

gazorby closed this Jan 30, 2023

gazorby force-pushed the main branch from 65da6f2 to f2cc503 Compare January 30, 2023 14:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Full support for nested pydantic, sqlmodel and ormar models #1637

Full support for nested pydantic, sqlmodel and ormar models #1637

gazorby commented Feb 10, 2022 •

edited

Loading

botberry commented Feb 10, 2022 •

edited

Loading

patrick91 commented Feb 10, 2022

codecov bot commented Feb 10, 2022 •

edited

Loading

gazorby commented Feb 12, 2022

alexhafner Feb 17, 2022 •

edited

Loading

alexhafner Feb 19, 2022

alexhafner Feb 17, 2022

gazorby Feb 18, 2022 •

edited

Loading

alexhafner Feb 19, 2022

alexhafner Feb 19, 2022

FraBle Feb 24, 2022

lovetoburnswhen Feb 24, 2022

FraBle Feb 25, 2022 •

edited

Loading

alexhafner Feb 18, 2022 •

edited

Loading

patrick91 commented Mar 5, 2022

alexhafner commented May 7, 2022

patrick91 commented May 7, 2022

alexhafner commented May 7, 2022

gazorby commented May 7, 2022 •

edited

Loading

patrick91 commented May 7, 2022

ElisaKonga commented Jul 26, 2022

khairm commented Dec 11, 2022

pranav-kunapuli commented Jan 19, 2023

cat-turner commented Jan 24, 2023

gazorby commented Jan 30, 2023

		sqlmodel = {version = "^0.0.6", optional = true}
		ormar = {version = "^0.10.24", optional = true}

Full support for nested pydantic, sqlmodel and ormar models #1637

Full support for nested pydantic, sqlmodel and ormar models #1637

Conversation

gazorby commented Feb 10, 2022 • edited Loading

Description

Pydantic

Ormar

SQLModel

Types of Changes

Issues Fixed or Closed by This PR

Checklist

botberry commented Feb 10, 2022 • edited Loading

Pydantic

Ormar

SLQModel

patrick91 commented Feb 10, 2022

codecov bot commented Feb 10, 2022 • edited Loading

Codecov Report

gazorby commented Feb 12, 2022

alexhafner Feb 17, 2022 • edited Loading

Choose a reason for hiding this comment

alexhafner Feb 19, 2022

Choose a reason for hiding this comment

alexhafner Feb 17, 2022

Choose a reason for hiding this comment

gazorby Feb 18, 2022 • edited Loading

Choose a reason for hiding this comment

alexhafner Feb 19, 2022

Choose a reason for hiding this comment

alexhafner Feb 19, 2022

Choose a reason for hiding this comment

FraBle Feb 24, 2022

Choose a reason for hiding this comment

lovetoburnswhen Feb 24, 2022

Choose a reason for hiding this comment

FraBle Feb 25, 2022 • edited Loading

Choose a reason for hiding this comment

alexhafner Feb 18, 2022 • edited Loading

Choose a reason for hiding this comment

patrick91 commented Mar 5, 2022

alexhafner commented May 7, 2022

patrick91 commented May 7, 2022

alexhafner commented May 7, 2022

gazorby commented May 7, 2022 • edited Loading

patrick91 commented May 7, 2022

ElisaKonga commented Jul 26, 2022

khairm commented Dec 11, 2022

pranav-kunapuli commented Jan 19, 2023

cat-turner commented Jan 24, 2023

gazorby commented Jan 30, 2023

gazorby commented Feb 10, 2022 •

edited

Loading

botberry commented Feb 10, 2022 •

edited

Loading

codecov bot commented Feb 10, 2022 •

edited

Loading

alexhafner Feb 17, 2022 •

edited

Loading

gazorby Feb 18, 2022 •

edited

Loading

FraBle Feb 25, 2022 •

edited

Loading

alexhafner Feb 18, 2022 •

edited

Loading

gazorby commented May 7, 2022 •

edited

Loading