Custom Ember Data the easy way [waynejohnson.net]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Custom Ember Data the easy way  ======

{{ :ember:ember-title-card.jpg?nolink&300|}}One of the best features of EmberJS is Ember Data. One of the most infuriating features of EmberJS is Ember Data. If you live in an ideal world, and your backend data format of choice is JSON:API, then you're on the happy path, and the world is your oyster.

But not many of us work on greenfields environments where JSON:API is the planned choice. The truth is that most applications have a history and therefore their own custom JSON format to suit.

It makes sense that EmberJS can support any type of data format using the ''RESTSerializer''. The problem is finding a nice easy guide that will walk the entire way through with actual examples of massaging some custom data.

Hopefully this guide help because I've wallowed in a pit here myself.

I want to state a couple of things up front to keep in mind as we work through to help:

  - RESTAdapter gives us the opportunity to massage data a little bit to fit our expected model, and to help Ember Data convert our data into JSON:API format behind the scenes (perhaps we're not using JSON:API, but EmberJS is internally).
  - Understanding the use of plurals versus singular in our code and data is important when massaging the data itself. This is the area that I think is the biggest challenge when starting out.
  
A couple of extra points before we get going. We're going to start with dummy data just to get something on the screen. While it's usual to give some sort of API to use, at times it's simpler to experiment offline and see a result with some basic JSON. 

The second point is that Mirage is sometimes recommended to start out mocking for Ember Data. But then, if knew what Mirage was or had experience with it, you wouldn't be reading this article. So let's skip both of those things for now.

As we go, we'll improve each step until we're got something more like a real application. 


===== Starting a new project ===== 

Create a new project with:

  ember new custom-data-workshop

Open the custom-data-workshop using VSCode, then open a new terminal.

I like to use a different port instead of the default 4200. You might be using the default for another EmberJS application. Therefore change package.json's start script as:

  "start": "ember serve --port 4300",

Now run as usual with:

  npm start

Open up in your browser to run your app. Finally, replace in ''app/templates/application.hbs'':

<code html>
<h1>Data</h1>

<p>Nothing</p>

{{outlet}}
</code>

We're good to go.

Oh just before that, would be nice to improve the styles just a little. Add this to the ''app/styles/app.css'':

<code css>
body {
    padding: 10px;
	font-family: sans-serif;
}

ul {
    border: 1px solid #BBBBBB;
    width: 260px;
    border-radius: 4px;
    padding: 7px 7px 7px 40px;
}

span {
    display: inline-block;
    background-color: black;
    border-radius: 25px;
    color: white;
    width: 28px;
	height: 28px;
	line-height: 28px;
    text-align: center;
}
</code>


===== Using Inline JSON =====

We'll start by returning some hard-coded JSON from a route for the default template to display. We already have an ''application.hbs'' template, so we will make an ''application.js'' route in which to load our data with:

  ember g route application 

Say no to overwriting the ''app\templates\application.hbs'' file.

Add a new ''model'' hook to create and return some JSON that will be used to display information about a game:

<code js>
import Route from '@ember/routing/route';

export default class ApplicationRoute extends Route {

    model (){
        const game = { 
            "id": 1,
            "title": "The Secret Of Monkey Island",
            "year": "1990"
        }
        
        return game;
    }
}
</code>  

Now to use those data values in our default ''application.hbs'' template. Replace the content with:

<code html>
<h1>Data</h1>

<ul>
    <li>{{@model.title}} <span>{{@model.id}}</span></li>
    <li>{{@model.year}}</li>
</ul>
</code>

Result:

{{ :ember:ember-data-template-output01.png?nolink |}}

Great! First result with some data showing on screen. On the downside, we are not using Ember Data just yet!

It is also good to see failure. Let's comment out the return in our model hook:

<code js>
    //return game;
</code>

{{ :ember:ember-data-template-output02.png?nolink |}}

Notice we get nothing rendered. We need to return the data object back otherwise the model hook considers it has received nothing. Best to uncomment that line again before moving.


===== Fetching Dummy JSON from a file =====

To start feeling more like a regular application we should ''Fetch'' our data rather than hard-code it within a route.

Create a JSON file in the ''public'' folder called ''game.json''.

Take the data from the model hook and copy it into ''game.json'' as:

<code json>
{ 
    "id": 1,
    "title": "The Secret Of Monkey Island",
    "year": "1990"
}
</code>

Change your application model hook to load the data from the file:

<code javascript>
import Route from '@ember/routing/route';

export default class ApplicationRoute extends Route {

    model (){
        let promise = fetch('game.json');
        return promise.then(function(response){
            return response.json();
        });
    }
}
</code>

Alternatively, you could use async/await syntax like this:

<code javascript>
import Route from '@ember/routing/route';

export default class ApplicationRoute extends Route {

    async model (){

        let response = await fetch('game.json');
        let data = await response.json();
        return data;

    }
}
</code>

Slightly easier on the eyes.

But either way, the data should be loading into the page from the file using fetch. Nice one.

For the sake of showing that this step can be difficult to get your resolved Promises right, here is some code that does not work for returning the data from a model hook:

<code javascript>
    model (){
        let promise = fetch('game.json');
        promise.then(function(response){
            return response.json()
            .then(function(game){
                return game;
            });
        });
        return promise;
    }
</code>

===== Fetching from a remote API =====

What about a similar API that is remote? Let's see how we go with that. At this point we need be online. I'll provide a RESTful test API at https://retrogames.waynejohnson.net/api

You can call the entire list of games with: https://retrogames.waynejohnson.net/api/games
Or you can call a single game with: https://retrogames.waynejohnson.net/api/games/{id}

First port of call, lets swap out the fetch call so that it makes a call to the API to get one record out of many:

<code javascript>
import Route from '@ember/routing/route';

export default class ApplicationRoute extends Route {

    model (){
        let promise = fetch('https://retrogames.waynejohnson.net/api/games/7');
        return promise.then(function(response){
            return response.json();
        });
    }
}
</code>

And that works too. But notice there is a subtle problem that will catch us up later. It's the id. 

{{ :ember:ember-data-template-output03.png?nolink |}}

<code json>
{
  "gameId": 7,
  "title": "Beneath a Steel Sky",
  "year": 1994
}
</code>

Notice that the ID is no longer displaying on our page. This is because we are expecting a property called ''id'' but our API is returning it as ''gameId''.

We could work around the problem by altering the JSON with:

<code javascript>
    model (){
        let promise = fetch('https://retrogames.waynejohnson.net/api/games/7');
        return promise.then(function(response){
            return response.json().then(function(json){
                json.id = json.gameId;
                return json;
            });
        });
    }
</code>

{{ :ember:ember-data-template-output04.png?nolink |}}

...but... as we get into collections (more than just a single record) and more complex results and patching, this is going to get very messy quickly. It's probably time to utilise Ember Data and have it do all the heavy lifting for us.

===== Fetching from an API using Ember Data =====

Now for the meatier stuff: actually using Ember Data to call the API for us and to bring in the data. For this, we'll start to use models and calls from our routes.

Begin by defining the model, which is the cornerstone of helping Ember Data help us. Create a model file with:

  ember g model game

This will create the ''game.js'' model file in the ''app/models'' folder. Change the content to be:

<code javascript>
import Model, { attr } from '@ember-data/model';

export default class GameModel extends Model {
    @attr('string') title;
    @attr('number') year;
}
</code>

You might observe that we don't have an ''id'' property. You should not specify an id in your Ember data models. More on this later.

Now because Ember Data prefers to consume JSON:API, we are going to need a custom Adapter to handle where the data is sourced from, and a custom Serializer to handle any massaging that we might need to fix up the JSON data we load in.

Make a new Adapter for our ''game'' model with:

  ember g adapter game

By default your new adapter is going to be a ''JSONAPIAdapter'' but that's obviously no good for our customised purposes. So let's change it into a ''RESTAdapter'' and add the ''host'' and ''namespace'' properties to point to our own API:

<code javascript>
import RESTAdapter from '@ember-data/adapter/rest';

export default class GameAdapter extends RESTAdapter {
    host = 'https://retrogames.waynejohnson.net';
    namespace = "api"
}
</code>

Let's now make our first attempt to load in data from the model hook in the ''app/routes/application.js'' route file. We need to bring in the ''store service'' by adding it to our includes, and also declare it inside the Route class:

<code javascript>
import Route from '@ember/routing/route';
import { inject as service } from '@ember/service';

export default class ApplicationRoute extends Route {
    @service store;
...
</code>


Replace the ''model()'' function with:

<code javascript>
import Route from '@ember/routing/route';
import { inject as service } from '@ember/service';

export default class ApplicationRoute extends Route {
    @service store;

    model (){
        let data = this.store.findRecord('game', 3);
        return data;
    }
}
</code>

There's some good news and bad news. If you check the Network tab, you'll notice there was a successful call to our API to retrieve back a record:

  200 GET 3	json	https://retrogames.waynejohnson.net/api/games/3

<code json>
{
	"gameId": 3,
	"title": "Twinsen's Odyssey",
	"year": 1997
}
</code>
  
Fantastic. That means that the adapter is doing it's job. If you want to know how the adapter works it's magic: it takes the ''host'' and ''namespace'' properties from the adapter settings, and the ''game'' from our ''this.store.findRecord("game", 3);'', and //pluralises// ''game'' into ''games''. Therefore:

  https://retrogames.waynejohnson.net + api + game + plural + index = https://retrogames.waynejohnson.net/api/games/3

Now the bad news... the page crashed with a blank page, and you will get an error in the console:

  Error while processing route: index payload.data is null

This is an indicator that we don't have a serializer for Ember Data that understands anything about our JSON structure. So we need to make a Serializer.


===== The Serializer and customising our data =====

We are going to spend some time here because this is the part that can be a complete mongrel to get right. Let's get started by making a serializer file with:

  ember g serializer game

Now with an empty serializer, we will get the following error because the serializer we created defaults to a ''JSONAPISerializer'':

  Error while processing route: index Assertion Failed: normalizeResponse must return a valid JSON API document

Let's turn it into a ''RESTAdapter'':

<code javascript>
import RESTSerializer from '@ember-data/serializer/rest';

export default class GameSerializer extends RESTSerializer {
}
</code>

Hmmm.. things appear to be worse:

  WARNING: Encountered "gameId" in payload, but no model was found for model name "game-id"
  WARNING: Encountered "title" in payload, but no model was found for model name "title"
  WARNING: Encountered "year" in payload, but no model was found for model name "year" 
  Error while processing route: index payload.data is null

Or

	Error: Assertion Failed: The 'findRecord' request for game:3 resolved indicating success but contained no primary data. To indicate a 404 not found you should either reject the promise returned by the adapter's findRecord method or throw a NotFoundError.

So far, Ember Data doesn't know how to convert our incoming data calls into something that suits our model.

A ''normalizeFindRecordResponse'' function is required in our serializer. Let's add a plain one that does little more than send our payload to ''super'' (super is in charge of converting our data into JSON:API behind the scenes internally):

<code javascript>
import RESTSerializer from '@ember-data/serializer/rest';

export default class GameSerializer extends RESTSerializer {
    normalizeFindRecordResponse(store, type, payload, id) {
        return super.normalizeFindRecordResponse(store, type, payload, id);
    }
}
</code>

Same error. Seems like we haven't progressed. Why don't we seem to getting anywhere? 

Let's take another look at the JSON that is coming back over the wire:

<code json>
{
	"gameId": 3,
	"title": "Twinsen's Odyssey",
	"year": 1997
}
</code>

Notice that the data has no root node. It's just an object. The serializer doesn't know what to make of it or which model it needs to map to. It's true we did tell the ''GameAdapter'' that we wanted a ''game'' object, but unless the server can show that it has returned one or more ''game'' data items, Ember will be upset. The server may have returned an error message, or worse, junk response from a crashed service. We wouldn't want Ember Data to proceed modelling on that "data".

The simple fix is to wrap the incoming data item with a ''game'' node. Let's update the normalizer function to be:

<code javascript>
import RESTSerializer from '@ember-data/serializer/rest';

export default class GameSerializer extends RESTSerializer {
    normalizeFindRecordResponse(store, type, payload, id) {

        const alteredPayload = {
            game: { payload }
        }
        return super.normalizeFindRecordResponse(store, type, alteredPayload, id);
    }
}
</code>

Hey we've progressed! A different error this time:

  Error while processing route: index Assertion Failed: You must include an 'id' for game in an object passed to 'push'
  
And you might be able to guess the reason for this? Remember how our JSON that was returned from the API had a ''gameId'', but our template ''application.hbs'' is expecting it to be ''id''? So we need to map the ID property. We can massage this within the ''normalizeFindRecordResponse'' function:

<code javascript>
import RESTSerializer from '@ember-data/serializer/rest';

export default class GameSerializer extends RESTSerializer {
    normalizeFindRecordResponse(store, type, payload, id) {
        const alteredPayload = {
            game: {
                id: payload.gameId,
                title: payload.title,
                year: payload.year
            }
        }
        return super.normalizeFindRecordResponse(store, type, alteredPayload, id);
    }
}
</code>

Woah! Hey! We got rendered output onto our page! Fantastic stuff. This means we have a complete working flow for our application for single records.

{{ :ember:ember-data-template-output05.png?nolink |}}

===== Collections of Data =====

Now that we have been successful with a single data record, and we understand that pluralisation and JSON root nodes can make all the difference... let's move on to working with multiple data records.

We'll need to change both our template page, and the route to expect multiple records. Let's start with the ''routes/application.js'':

<code javascript>
import Route from '@ember/routing/route';
import { inject as service } from '@ember/service';

export default class ApplicationRoute extends Route {
    @service store;

    model (){
        let data = this.store.findAll('game');
        return data;
    }
}
</code>

Oooh, a ''findAll'' call. You'll notice we don't have an error, but even still: our data doesn't populate on the page. However... the Network tab will show that the call successully loaded all our records because the GameAdapter constructed a URL like this: ''https://retrogames.waynejohnson.net/api/games''. 

How? Again, the adapter takes the ''host'' and ''namespace'' properties, and the ''game'' from our ''this.store.findAll('game');'', and //pluralises// ''game'' into ''games''. Therefore:

  https://retrogames.waynejohnson.net + api + game + plural = https://retrogames.waynejohnson.net/api/games

The trick to all this Ember Data stuff is to be aware of what convention needs to be a ''single'' and what needs to be a ''plural''. After that, everything is smooth sailing.

There are a bunch of warnings in the developer console like this:

  WARNING: Encountered "0" in payload, but no model was found for model name "0".

This means Ember Data is trying to parse each record, but the RESTSerializer doesn't know what to do with a collection of ''game'' objects coming in. We better teach it by supplying a new ''normalizeFindAllResponse'' function into the RESTSerializer:

<code javascript>
	normalizeFindAllResponse(store, type, payload) {
		return super.normalizeFindAllResponse(store, type, payload);
	}
</code>

The same warnings occur because we did nothing to massage in the incoming rows. Let's try wrapping the collection of JSON objects with a root node:

<code javascript>
    normalizeFindAllResponse(store, type, payload) {
        const newPayload = {
            games: payload
        }
        return super.normalizeFindAllResponse(store, type, newPayload);
    }
</code>

Notice the plural ''games'' node for a collection? This is good. The warnings are gone, however and we're back to an error:

  Error while processing route: index Assertion Failed: You must include an 'id' for game in an object passed to 'push'
  
Looks like we need to ensure we massage the ids for each row coming in. Change the RESTSerializer to do this with:

<code javascript>
    normalizeFindAllResponse(store, type, payload) {

        const newPayload = payload.map( g => {
            return {
                id: g.gameId,
                title: g.title,
                year: g.year
            }
        } );

        const wrappedPayload = {
            games: newPayload
        }
        
        return super.normalizeFindAllResponse(store, type, wrappedPayload);
    }
</code>

That's fixed the data but we can't display it just yet.

That code was a little long too, and could be optimised. But you can easily follow what's happening here. There is an easier way to do this for collections: you can add a ''primaryKey'' property to the RESTSerializer:

<code javascript>
import RESTSerializer from '@ember-data/serializer/rest';

export default class GameSerializer extends RESTSerializer {
    primaryKey = 'gameId'; //<------ note!!

    normalizeFindRecordResponse(store, type, payload, id) {
        const alteredPayload = {
            game: {
                gameId: payload.gameId, // <- Set prop to gameId again. primaryKey will sort this out for us!
                title: payload.title,
                year: payload.year
            }
        }
        return super.normalizeFindRecordResponse(store, type, alteredPayload, id);
    }

    normalizeFindAllResponse(store, type, payload) {

        const wrappedPayload = {
            games: payload
        }
        
        return super.normalizeFindAllResponse(store, type, wrappedPayload);
    }    
}

</code>

The errors and warnings are gone, and so therefore all that is left is to update our ''application.hbs'' template to ensure each record is rendered:

<code javascript>
<h1>Data</h1>
{{#each @model as |game|}}
<ul>
    <li>{{game.title}} <span>{{game.id}}</span></li>
    <li>{{game.year}}</li>
</ul>
{{/each}}
</code>

And viola! We have a collection of games rendering in a list. And I think we'll stop there, that's plenty enough. But there's much more to learn, such as relationships, also POSTs, PUTs and PATCHes weren't covered. So have fun experimenting!

{{ :ember:ember-data-template-output06.png?nolink |}}

===== Thanks =====
Thank you to Latha.K & Andrew.W for proofing and tips, and Julien.P for wallowing with me with every possible permutation of Promises.

===== Further reading, tips, thoughts =====

  - https://emberigniter.com/fit-any-backend-into-ember-custom-adapters-serializers
  - https://javascript.info/promise-chaining
  - https://guides.emberjs.com/release/routing/specifying-a-routes-model
  - https://emberigniter.com/render-promise-before-it-resolves
User Tools

Site Tools

Page Tools
