1. Today we’re going to talk about scaling your team with GraphQL, and why relationships matter at all points in the process - both the relationships within your data, and the relationships within your organization and teams. Much of this content comes from my own recent learnings, and specifically from attending GraphQL Conf 2019 which just wrapped up in Berlin.
2. I’m Joel, I’m a Senior Software Engineer at Skookum, which is an impact focused strategy, design, and development agency in Charlotte, NC. You can find me on Twitter @JoelSocialized, or reach me at my email hello@joel.software.
This photo is particularly important to me, and I’d like to take a moment to talk about why and the value of conferences. If you’re not familiar with the GraphQL Community; on the left is Lee Byron, one of three co-creators of GraphQL and recently coronated Chair of the GraphQL Foundation governing board. In the center is Sashko Stubailo, a long-time contributor to the GraphQL community and someone who I personally learned a lot from when I was first getting into GraphQL in 2016/2017 while he was at Apollo and posting a lot of content on the subject. And lastly, on the right, that’s me. I think I’m wearing that shirt right now…
This photo is important to me not just because of the well known people in the front,though that’s an amazing part of going to a conference, it’s the people in the background who I recognize from the time we spent together during the talks and after-hours at biergartens and after parties talking about our work, sharing stories, and learning from one another.
If you’re thinking about going to a conference, do it! Whether or not it’s GraphQL Conf, choose one that aligns with your personal interests and professional goals and go! Conferences are more than just the learnings from the stage, may conferences including GraphQL Conf are live-streamed for free and the talks are posted online later. You can watch this content at 2x speed in the comfort of your own home if your primary goal is learnings from the stage, but I say it's much more than that. Go, and build relationships with the people you meet there – that is where the true value of a conference is found.
3. So – have you heard about GraphQL? I hope you have, and that’s why you’re here right now listening to me share my learnings.
4. You may be at any number of points in your GraphQL journey. You may be API Creators, Consumers, or just getting familiar with the concepts. You may feel confident in your knowledge, or confused by the ecosystem. Wherever you’re at, I’d like to start by talking about the specification so we can start with some shared concepts and understandings.
5. This comes straight from the open source specification, the current working draft available on GitHub and at https://graphql.github.io/graphql-spec/.
If this is enough for you, I’ll take volunteers to give the rest of the talk, if not let’s dive in a bit and work through these concepts.
6. So, what is GraphQL?
7. GraphQL is an agreement (hint: relationship) between the client and the server
8. GraphQL is a query language for APIs
9. GraphQL is not a database query language
10. GraphQL doesn't specify where or how to store your data
11. GraphQL doesn't specify how to execute retrieval of your data within your various services Certainly “resolvers” are the functions that run in order to retrieve data from your various services, but the method by which you execute that retrieval within those services, or the protocols you use to communicate with them, is explicitly not defined by GraphQL.
12. Still have some questions? Let’s refactor your mind from REST
13. I’m going to use REST as a framework for further discussion, as I am assuming most of you are more familiar with REST as a mental model.
I love this diagram from Lachlan Young, a Software Engineer from Australia who spoke at GraphQL Conf and was a delight on and off stage.
The concepts on the left, HTTP methods or verbs for interacting with APIs may be more familiar to you - though I remember when I first cracked open the 403 pages on RESTful Web APIs from O’Reilly, and I’m still uncertain about some of those concepts or when exactly I want to use some of these verbs, as you may be also.
Regardless, if you want to GET data in GraphQL it is called a Query. If you want to change data in GraphQL it is called a Mutation. If you want to subscribe to the changes of that data, is is called a Subscription.
14. So, what does this look like?
15. Let’s consider a REST API with the following resources.
You have a list of projects, and the ability to look up individual projects by their name.
You also have a list of contributors, and the ability to look up contributors based on their id.
In our example, this is an API with 4 endpoints, and likely more for associated data.
16. In GraphQL this is a single endpoints, and you can do all of the previous things through a GET request where you specify entities in a query param.
17. GraphQL servers also use POST which is particularly valuable for running multiple operations and mutations with variables. At the end of the day, GraphQL APIs are still using HTTP methods GET and POST primarily - though you won't find those methods defined in the spec.
18. Now you might be wondering, what did you mean by entities?
19. Well, one way you may start to defined your entities, and their relationships, is through types.
Here we have the Project type, and it has a Name which is a String, a Tagline which is also a string, and n contributors.
20. The contributors are our first relationship!
21. Now let’s say we wanted to ask for a specific project, which is called “GraphQL” and we need the tagline of that project for our application.
You can see here we’re using a query, passing a name, and requesting tagline.
This is defining the shape and content of the data we want to get back! It looks a bit like JSON...
23. What we get back from the server is JSON! And the data is in the shape we previously described.
24. Now we may also want to get a list of contributors to our project, and since we defined that relationship previously it’s easy to request the data we need by adding contributors, and specifying username as the data to return.
25. Now the JSON that is returned includes contributors and their username!
26. So, how does this help?
27. Likely, request waterfalls are something you’ve experienced before, wherein you have to make one request in order to make a subsequent request based on the returned data, in order to make another request… In GraphQL you can often request all of the data you need in a single query.
Overfetching is when you request data from an endpoint that returns the data you need along with data you do not need. Many RESTful endpoints work this way - if you want the username, you may have to request the whole user object. There was some research that was recently made available through Cornell which postulated a conversion to GraphQL from REST could reduce total data-field requests by 94%, and total data transfers by 99%. You can go read the HackerNews post on this research to see how people responded to those claims, but what I’d like to leave with you is the sense that significant reduction in data retrieval and data transfer can be achieved with GraphQL. Data transfer is expensive, both for your services, networks, and certainly for you clients on their various devices and internet connections.
Underfetching is when you aren’t able to get what you want from the available endpoints, and may have to go to tertiary data sources to get related data. You may also be dealing with overfetching of superfluous data, even while underfetching by not getting what you need from a single source or single request. And certainly, with GraphQL you’re getting Type-safety out of the box which allows you to reason about the shape of your data, and the content of your responses and mutations.
28. Documentation - this was particularly exciting for me when I first began working with GraphQL as I had recently spent a significant amount of time documenting API endpoints. Out of the gate with GraphQL and features like introspection you’ll get documentation. Consumers of your API need to know what data is available, and what manipulations are possible. GraphQL Schema gives you that data by default. There are also may tools available from the community to help you add to the documentation.
Code generation is one of the holy grails of software engineering - not having to software engineer! Given the type-safe quality and introspection, you can generate boilerplate code for common interactions with your data, so you can direct your focus on building edge case data requirement operations instead of common CRUD operations.
Because of GrahQL’s schema definitions, you can also quickly mock our data responses for your tests – and since I know we all right tests for our code all the time – this is a huge productivity win!
Last but certainly not least, Query Validation means you can get instant feedback on the queries you’re writing well before you send them to the server or have to parse an error message. This feature comes primarily from the tooling built around GraphQL, but is ubiquitous in most implementations.
29. Just as important as how GraphQL can be helpful, is knowing what GraphQL does not natively help with.
30, 31. Caching, Rate Limiting, Authorization, Load Balancing, CORS, Tracing & Analytics, File Uploads...
You will certainly be able to find guidance on these and other topics from the GraphQL community and the many reference implementations and open-source tools. However, you won’t find these defined in the spec, and you may not find a clear and final answer on how to handle these in your specific implementation.
32. You might be asking why not?
33. In some ways, GraphQL follows the Unix philosophy - this is something that Dan Schafer shared at the opening of GraphQL conf.
34. “There are a lot of questions that [GraphQL] very specifically does not answer… that’s because [these questions] were already solved by FB in 2012” - Dan Schafer (GraphQL Co-Creator) GraphQL before GraphQL — Dan Schafer
GraphQL was made to compose with these solutions
35. These are still important considerations! You will still have to account for these concerns in your implementation, and you will find help in the GraphQL ecosystem.
36. GraphQL as a spec is “small”, and should remain a thin layer in your stack
37. What might a GraphQL architecture look like?
38. Let’s say you have some clients, mobile and desktop, and a database. I know many of you are probably wishing your situation was this simple.
39. If this is your setup, you can just put GraphQL in front of the database, and connect directly to your data through resolvers and an ORM or even raw database queries.
40. Or maybe, and more likely, your implementation looks a little more like this - where you have microservice(s), database(s), and other third party APIs or legacy RESTful APIs you still maintain and interface with.
41. GraphQL can be the gateway to all of these individual services and data layers.
42. GraphQL can be where you DTR, define the relationships, and create a single graph of the necessary data for your clients.
43. So, you want to build a GraphQL API - Let’s talk about principles.
Given that many of you are working in different languages, toolchains, implementation constraints, and organizational strategies, let’s talk about building a GraphQL API through principles - ideals that you and your team can strive for as you make decisions and compare these concepts to your individual constraints.
44. We’ll do this by looking at Principled GraphQL, a resource shared by the Apollo organization based on their wealth of experience building and hosting GraphQL APIs and associated tooling.
Apollo estimates that their software is used in over 90% of existing GraphQL implementations. Whether or not that’s an accurate number, Apollo has been around for a long time actively contributing to the GraphQL ecosystem, and they have a lot of value to bring to this discussion.
45. One Graph - Your company should have one unified graph, instead of multiple graphs created by each team.
46. Federated Implementation - Though there is only one graph, the implementation of that graph should be federated across multiple teams.
47. Track the Schema in a Registry - There should be a single source of truth for registering and tracking the graph.
48. Abstract, Demand-Oriented Schema - The schema should act as an abstraction layer that provides flexibility to consumers while hiding service implementation details.
49. Use an Agile Approach to Schema Development - The schema should be built incrementally based on actual requirements and evolve smoothly over time.
50. Iteratively Improve Performance - Performance management should be a continuous, data-driven process, adapting smoothly to changing query loads and service implementations.
51. Use Graph Metadata to Empower Developers - Developers should be equipped with rich awareness of the graph throughout the entire development process.
This one made me chuckle when I first read it, the phrasing is so powerful!
Essentially, ensure your developers can easily access Graph information, have good tools and use them.
52. Access and Demand Control - Grant access to the graph on a per-client basis, and manage what and how clients can access it.
53. Structured Logging - Capture structured logs of all graph operations and leverage them as the primary tool for understanding graph usage.
Logging should be one of the first things you consider, not one of the last. In order to understand your graph usage and iteratively improve performance (Principle #6) you’re going to need this information.
54. Separate the GraphQL Layer from the Service Layer - Adopt a layered architecture with data graph functionality broken into a separate tier rather than baked into every service.
55. The most important thing about GraphQL is...
With all of these things in mind, and considering all of the benefits of GraphQL - what do you think the most important thing about GraphQL is? I have my own opinion, which I will share. But I’d like you to consider for a moment what is most impactful for you and your team.
56. I believe the most important thing about GraphQL is the schema(.graphql), with this you unlock incredible features and potential for you and your teams.
57. You will have many data stores and services but one graph
58. So with many data stores, you might be asking how will I generate this schema?
59. When you read GraphQL Principle #2, you’ll probably see the value in federating the schema authorship as well.
60. What if I want to distribute my schema authorship across my services?
61. This. Is. Difficult.
62. This. Is. Confusing.
63. Schema Stitching, Schema Federation,
GraphQL Gateway, GraphQL Modules,
Namespaces, Schema Delegation,
Schema Composition...
64. Point-point communication without GraphQL as a requirement can be optimized in a way that is free of client constraints and use cases
65. Use cases are the relationship between features and the necessary data.
66. Don’t design your schema based on your services architecture.
This is something that your API consumers do not need to, or want to understand.
And if you properly abstract your services architecture, you’re free to change your services without making your data graph obsolete and outdated.
67. You may be familiar with this pattern originally made famous by SoundCloud, Phil Calcado (not at Meetup) recently revisited this concept in contrast to GraphQL and found that these concepts are not mutually exclusive.
68. If you do distribute your schema, “stitch” statically, track changes.
I have to take a page from Marc-Andre’s book here (not actually, though he is writing a book!) and recommend you not stitch dynamically. But ideally, distribute your execution, not your schema
69. Let your individual service teams work on optimizing their domains.
70. If this is the kind of content that is interesting and useful to you and your work, I highly recommend these talks from GraphQL Conf.
71. So, you want to use a GraphQL API. Let’s talk about tools.
72. You’ve got a schema, how do you use it?
73. I quite like graphql-cli, an OSS offering form the Prisma OSS team.
74. All you need to do to get started with graphql-cli and your project, is to run `graphql init`. The CLI will ask you to answer a few questions, and then generate a .graphqlconfig file for you.
And don’t worry, if your GraphQL API requires something like an Authorization header, it’s easy to configure that as well.
75. Once you have a configuration file, it’s as easy as running `graphql get-schema` to hit your API and create a schema.graphql file locally in your project.
With the schema file saved in your project, a whole world of tooling and validation will be available to you.
76. Maybe you’re using VSCode, in which case you may want to try the GraphQL extension from Prisma. This extension gives you syntax highlighting, validation, go-to definitions, and all sorts of productivity wins.
77. In this example, I’ve asked for a number of fields when creating a “Beans” fragment, however the existing type does not have a `roastTime` property and i’m getting immediate feedback about that at the authorship stage, rather than in a build or runtime error.
78. In this example, I’ve tried to run a `createBean` mutation when the actual mutation is call `createBeans`, and i’m getting immediate feedback and correction recommendations before I ever hit save.
79. Another great feature available as an easy to install plugin for the CLI is graphql-voyager. Voyager allows you to visualize your data schema and relationships between types. Rather than spelunking into your schema or navigating docs to see what is available, you can explore the data types visually and quickly find information on data models and operations.
80. Consider using an IDE like GraphQL Playground which is available directly from the CLI by running `graphql playground` as well as a desktop app you can install. Using an IDE makes storing query variables, browsing docs, testing subscriptions, and more pretty simple and fun.
81. If you’ve spent any time with GraphQL query editing in the browser, you might have seen Graphiql before - definitely one of the best ways to get familiar with an API and test queries.
82. My favorite thing about Graphiql, and my favorite way to write a query, is the relatively new explorer plugin. In this example I’m generating a GraphQL query to the GitHub API by selecting the data I want which is displayed as a series of checkboxes and expand menus. All you have to do is select the data, even fill out query variables and sorting without ever writing a line of code. Validate the result, and save the query in your application.
83. If you’re working on a newer project, you may be in the position to choose a graphql client. These are just some of the options available.
Apollo is certainly one of the most popular options, and comes out of the gate with things like local caching, and optimistic updates, though I have found Apollo client to be a bit verbose at times.
Relay from facebook is also quite popular if you’re working in React, and comes with powerful mocking capabilities and custom GraphQL directives.
If you’re working with AWS AppSync you might find yourself using the Amplify client, though it doesn’t have some of the bells and whistles of the other options.
Finally, if you don’t need some of the custom features of these other clients or aren’t interested in learning their intricacies, try out graphql-relay from Prisma which is a minimal GraphQL client that will get you up and running quickly.
84. Generate mock data! The graphql-cli also is supposed to get a graphql-faker plugin soon, which should make generating mock data even easier.
85. Generate docs!
There are plenty of open source tools to help you generate API documentation in addition to the existing schema documentation available for GraphQL, check out the awesome-graphql repo on github for some options. You can do thinks like generate markdown files or choose a hosted solution.
86. Consider maximizing your benefits with a type-safe codebase
87. For me, one of the big takeaways from GraphQL conf, and prisma day which I also attended was the assumptive tone from many of speakers that you would want a type safe codebase. I’ve long been a fan of TypeScript for example, but have found it to be somewhat polarizing as a topic. At GraphQL Conf and Prisma Day there was plenty of support for type safety in JavaScript whether with TypeScript or Flow.
88. If this is the kind of content that is interesting and useful to you and your work, I highly recommend these talks from GraphQL Conf.
89. So, have you heard about GraphQL?
90. At the close of the conference, Lee Byron noted the value of better abstractions, better syntax, and better mental models over the last 30 years of the web - the assertion being that GraphQL would be one of the better mental models that could take us forward.
91. How about the relationships that brought us here?
92. How about the relationships that will take us there?
93. Relationships thrive when they have shared language & shared goals
94. GraphQL can be the shared language for your teams as they drive towards shared goals.
95. GraphQL is an agreement (relationship) between the client and the server
96. It’s an agreement between your teams.
97. So don’t just think in terms of data graphs think in terms of relationships
This is the thought I’d like to leave you with.
98. Thank You
Click here to get notified when I post new content