Models or Types

The main crime of the old-school (backend-driven) web dev is their substitution of type concept with model concept. I point to old-shool instead of dynamic typing because of JAVA. They remain the core promoters of ORM mythology up to this day. But first things first.

Model is a Type

What is a Model? It's a declarative description of something, stored in DB. Basically, a type of it. All SQL implementations have notoriusly bad and limited column types. Boolean is represented as TINYINT(1) and is returned to the consumer as 1 or 0 because a driver has no idea it was meant to be Boolean. Sometimes dates are also returned as String even if native Date is present. JSON fields are a must and their support is beyond criticism. As a rule of thumb, you'll need to "parse from" and "format to" SQL. There is also an issue of SQL injections.

Bottom line: it's really hard to stick with pure SQL. So you probably need some sort of a wrapper. And here the story begins.

What's wrong with Model? First – you can have second database. Oh wait... second ORM?! Core datastructures described twice. In different syntaxes. Good luck!

Second – what's about frontend? You will describe the same structure twice – for server and for client. In different languages. Or in different syntaxes, in the best case of fullstack JS.

Good code is declarative so types (with related helpers) can easily make up to 30% of the codebase. Dependent Types will increase this number. "Every bug is a type issue" is real and coming. And all this huge volume is now a subject to the code rot.

Take any ORM, e.g. Bookshelf.

let User = bookshelf.Model.extend({
  tableName: 'users',
  posts: function() {
    return this.hasMany(Posts);
  }
})

We describe User type in the language of "models".
But we are very, extremely, outrageously limited.

Models N × M

Real datastructures tend to look like this:

let T = require("tcomb")

// Main fields (visible for all roles, except "visitor")
let PublicUser = T.struct({
  id: Id,
  role: UserRole,
  username: T.String,
  fullname: T.String,
  blocked: T.Boolean,
  rating: T.Number,
})

// Protected fields (visible for paid users and owner)
let ProtectedUser = PublicUser.extend({
  phone: T.maybe(T.String),
  email: T.maybe(T.String),
})

// Full "model" (all DB fields) (visible to owner / admins)
let User = ProtectedUser.extend({
  balance: T.Number,
  createdAt: T.Date,
  editedAt: T.Date,
  paidUntil: T.maybe(T.Date),
})

// Form type (creating / editing)
let UserForm = T.struct({
  username: T.String,
  fullname: T.String,
  phone: T.maybe(T.String),
  email: T.maybe(T.String),
})

Could you describe all this with any ORM? No way.

They will allow you to have a single model-type while there are a multitude of them in reality. You can ignore this fact, "solve" it imperatively, redeclare things with different syntaxes and limitations. Once you're reach this point – you're screwed.

SQL fallbacks

Most ORMs authors are very proud of how they leave raw SQL fallbacks to us, pure mortals. – Even ORM can't cover 100% usecases – they kindly accept.

But what happens when you fallback to raw SQL? All those quirks we mentioned come back shamelessly! Some of them, like SQL-injections, can be solved by lower layers (if we're lucky enough and ORM is built on top of query builder).

Some of them, like parsing issue – can't. So you either write imperative code being prepared for a bug zoo. Or you start to grow up yet another wannabe-type substitution layer, right next to model declarations...

Validation

The question of types is tightly coupled with validation. Non-dependent typesystems can't express repeatPassword == password and similar edge cases. A few of them. Validation rules follow type rules with striking similarity. And add... customizable error messages. Not closely enough to satisfy the duplication of concerns. Those messages can easily be built on top of types (if types are first-class).

So further you will use awkward libs like JOI repeating yourself once again. And once more. Fighting with unsync issues between models and validation rules. Loosing to them. Those who ignore the reality deserve their doom.

So what's a reality? Types. Everything has a type. Everything. The composite "address" field is not an entirely different thing than "user". They have the same structure. They should be described with the same language.

-- Realm of MODELS --
user       -- Model
  address  -- JSONField
    street -- StringField

street = user.address.street -- String

-- Ladies and Gentlemen! Let me proudly present our special guest of Complexity!

----------------------------------

-- Realm of TYPES --
user       -- Struct
  address  -- Struct
    street -- String

street = user.address.street -- String

-- So boring. Too simple!

You need types, not "models", my friend.

Alternatives

But I can't use pure SQL?!

Just don't use ORMs. There are so many alternatives: NoSQL, DAO, query builders with custom parsing. Everything that does not try to substitute typesystem will be a better choice. Currying and functional programming will help you, as always.

// SQL + Knex
let users = map(parseAs(User))(await db("users").select("*"))

// RethinkDB + RethinkDB pool
let users = conn.run(await table("users"))

Avoid things that poison your mind. Some bad ideas are very sticky. OOP, REST, ORM... The subjective perception of importance of something goes up with time invested (wasted).

Models are real but they are only a fragment of reality. We need to see the whole picture. Describe reality with tools expressive enough to represent it's natural complexity. Instead of squashing it into a vulgar and limited vision of ORMs. Here is the real "mismatch". With ineviatable consequences of incidental complexity, bugs, losses and dispair.

Backend crime

I could continue with a technical criticism like this or this but it would just dilute the main point. I used DoctrinedORM, PropelORM, SQLAlchemy, Peewee, Bookshelf, etc. I even wrote a couple of my own in Python and PHP. Every time it was the same experience in the end.

I claim that is the crime of backend devs back in 200x. They didn't have decent NoSQL solutions back there. They wasn't aware of alternatives. They were required to get things done. Some excuses can be applied.

But then they grew up to believe their own hack. That their crutches over SQL implementation flaws are useful on themselves. Instead of making push to improve SQL drivers (cutting down the reasons for ORMs one by one) they chose to bend. And contaminated thousands of minds.

The sickness spreads. There are ORMs ODMs for Mongo. For RethinkDB. Being nothing else than awful typesystem replacements. Typesystems for poor.

And (in the end) there are people who brought "Unit Of Work" pattern into the web. Uncontrollable, implicit caching of every query in non-interactive environment. Sociopathy. The model-based mindview is obviously dangerous.

Joins

There are more angles worth mentioning. A JOIN concept. SQL joins smash the data together, providing new obstacles for the typing.

SELECT * FROM users
INNER JOIN comments
ON comments.user_id = users.id

will return a list of rows where user data and comment data are interleaved into mess. That's probably another problem ORM were meant to solve. And another actually SQL-specific problem.

You don't want to describe a special JOIN-related type? Well, you'll have to, because the types you have don't cover merged case.

Now see how joins are implemented in RethinkDB. The resulting sequence contains documents with left and right parts which you can zip (if you want) or handle in any other way. You know which fields come from which table.

{
  "left": {
    "id": "543ad9c8-1744-4001-bb5e-450b2565d02c",
    "text": "You can fool some of the people all of the time, and all of the people some of the time, but you can not fool all of the people all of the time.",
    "userId": "064058b6-cea9-4117-b92d-c911027a725a"
  },
  "right": {
    "id": "064058b6-cea9-4117-b92d-c911027a725a",
    "fullname": "Abraham Lincoln"
  }
}

I can't stress enough how much smarter this is. How much more convenient and user-friendly. People who say NoSQL movement "is just trendy" have vision problems.

REST

Now to the REST thing. Shaping the context of 200x software catastrophy we can't omit this idol, can we?

Consider the following ways of making new User.

POST /users     {username: "jack", ...} -- body shouldn't have id (because autogen)
PUT  /users/:id {username: "jack", ...} -- body shouldn't have id (because conflict)

The data which you send in body should not have id. So you should grant one more type for this (see UserForm above).

While you can technically have only User and drop UserForm it will make things dramatically harder. You'll need to manually manage all fields. Imperatively.

  1. Protect against id spoofing
  2. Protect against balance increasing
  3. Protect against role switching
  4. ...

And so on and so forth. While with adequate typing it's:

router.post("/api/users/role/:role",
  KoaValidate({
    "params.role": UserRole,
    "body": UserForm,
  }),
  function* (next) {
    ...
  }
)

Declarative. Simple. Without duplications. Now go and do that with Knex + Bookshelf + Joi + whatever frontend validation library.

What if you had a very simple case and wanted to use only one type? The obvious way of doing would be:

PUT /users {id: "1", username: "jack", ...}

But it's nnnoot rrreestful! Roy Fielding personally prohibited this. And here, in IT, we are very sensitive to guru's opinions. Now seriously, why someone has a right to dictate you such details? Why, on earth, someone grabs app-specific things not even in library. In protocol! And call it a day.

Who are those people? What are they doing in industry? With industry? But this is a subject for another serie.

I really hope GraphQL will undermine REST hard enough. With all their flaws Facebook team deserves respect if only for their bravery. They're basically trying to "destroy" three holy grails of the Web: CSS, templates, and REST. The deed with no precedents. Can't say I will miss the last two.

Perspectives

So what is our perspective? Can we expect of this "model madness" to gone someday?

Yes, but we need first-class types for that. As far as I know, TypeScript does not provide an access to the type information. It technically could be exposed via hooks or plugin system but it isn't there yet. Some people asked about this stuff here and there but it's not gaining traction in general.

There are brave individual atttempts worth attention. Without hooks (or other kinds of help from compiler) they have to reimplement a lot of parser functionality. But it (already) works. And this is huge.

Addon I

@GiulioCanti points out there are some interesting Flow utils which can possibly help to merge static and dynamic realms.

See this and this for more details.

Addon II

People ask why JOI is awkward. How else can I call officially backend-only validation library? Last time I checked, it added 1.2 Mib(!) to the bundle. Bury it.