DEV Community: Nicolas Takashi

Platform Monitoring - Primeiros Conceitos

Nicolas Takashi — Thu, 02 Dec 2021 11:45:10 +0000

Bem, há muito tempo não compartilho minha experiência como engenheiro de software trabalhando em uma equipe de observabilidade, mas adivinha quem está de volta?

É importante lembrar que, antes de pularmos para tópicos avançados, quero cobrir os fundamentos e garantir que engenheiros de qualquer nível possam consumir esse conteúdo. Hoje, vou falar de monitoring como um dos pilares de observabilidade.

O que é monitoring?

Monitoring fornece visibilidade detalhada de desempenho, disponibilidade, experiência do usuário e utilização de recursos, ajudando-nos a fornecer um desempenho de plataforma consistente com um baixo MTTR (Mean Time To Resolve).

Por que monitoring é crítico?

Quando uma plataforma apresenta problemas de desempenho ou não está disponível, a empresa que possui essa plataforma corre o risco de perder clientes. As ferramentas de monitoring fornecem insights de desempenho e disponibilidade em tempo real que permitem às equipes reagir rapidamente quando surge um problema.

Por que uma plataforma deve usar uma ferramenta de monitoring?

Uma ferramenta de monitoring pode oferecer algumas vantagens para uma organização, como:

Reduzir MTTR

Uma ferramenta de monitoring ajuda os engenheiros a entender como um dia normal se parece, usando métricas de desempenho para definir suas linhas de base, definir alertas adequados com base nessas métricas e identificar a causa raiz dos problemas de desempenho ou disponibilidade.

Aumento da receita

Uma ferramenta de monitoring ajudará as equipes de engenharia a identificar rapidamente um problema crítico de desempenho ou disponibilidade que frustra os clientes da plataforma. Clientes frustrados começam a encontrar outras plataformas para executar suas necessidades, o que impacta diretamente sua receita.

Reduzir custos

Uma ferramenta de monitoring ajuda uma organização a entender como é um dia normal, verificando quanta memória e CPU um aplicativo precisa e verificar se os recursos de uma máquina virtual forem superdimensionados.

A equipe de engenheiros pode ajustar a utilização de seus recursos, economizando dinheiro em infraestrutura com essas informações.

Estratégias de monitoramento

Existem dois tipos de estratégias de monitoramento disponíveis, monitoramento de caixa aberta, conhecido como monitoramento de caixa branca, e monitoramento de caixa fechada, conhecido como monitoramento de caixa preta.

Ambas as estratégias são importantes e fornecem visibilidade de plataforma diferente sobre desempenho e disponibilidade.

Monitoramento de caixa aberta

O monitoramento de caixa aberta oferece uma visão abrangente do serviço ou aplicativo. Para entender melhor isso, vamos usar o seguinte serviço em python como exemplo.


from flask import Flask
from prometheus_client import make_wsgi_app
from werkzeug.middleware.dispatcher import DispatcherMiddleware

app = Flask("apione")

app.wsgi_app = DispatcherMiddleware(app.wsgi_app, {
    '/metrics': make_wsgi_app()
})

@app.route("/")
def monitoring_ready():
    return "Hi there..."

app.run(host="0.0.0.0", port="3000")

Esse exemplo de serviço simples expõe o estilo de métricas do Prometheus no endpoint /metrics. Instrumentar um serviço ou aplicativo usando o cliente Prometheus não é nada complexo, mas se você não estiver familiarizado com isso, recomendo que dê uma olhada na documentação oficial.

De agora em diante, quando invocarmos o endpoint de métricas, estamos obtendo a página de métricas do Prometheus que retorna um monte de métricas padrão com base em sua tech stack.

Dado que é um exemplo simples que retorna apenas as métricas padrão do Python, podemos começar a adicionar métricas personalizadas para esse aplicativo monitorando o serviço com precisão.

Monitoramento de caixa fechada

O monitoramento de caixa fechada, conhecido como monitoramento de caixa preta, é uma estratégia para monitorar serviços de fora usando sondas artificiais. Isso significa que não aproveitamos nenhuma informação interna do serviçóalém do resultado de uma probe artificial.

O que significa probes artificiais?

Os probes artificiais podem ser uma solicitação HTTP, soquete TCP ou execução de script para executar uma tarefa específica no serviço ou aplicativo.

Por que o monitoramento de caixa fechada é necessário?

O monitoramento de caixa fechada é fundamental para ajudar as organizações a implementar o monitoramento sintético e simular o comportamento do usuário periodicamente, e essa é uma estratégia que muitas empresas já utilizam.

Para obter mais informações sobre o monitoramento sintético e como o monitoramento de caixa fechada ajuda nessa implementação, verifique a documentação oficial do Grafana.

Conclusão

Como podemos ver, monitoring é crucial para qualquer organização, não importa seu tamanho. A organização responderá rapidamente a incidentes, escalará seus negócios e aumentará a felicidade do cliente usando estratégias de monitoramento de caixa aberta e caixa fechada. Implementações de monitoramento mais sofisticadas podem ajudar a organização a prever incidentes e abordar de forma proativa os problemas de desempenho e confiabilidade.

Como você pode imaginar, existem várias ferramentas de monitoramento disponíveis no mercado, tais como:

Zabbix
Nagios
Prometheus

Sempre utilizaremos o Prometheus como exemplo nessas postagens, pois é uma tecnologia cloud-native que nos permite trabalhar com qualquer tipo de infrastructure.

Monitoring é um assunto amplo e profundo que abordaremos passo a passo nas próximas postagens. Eu espero que você tenha gostado. Fique à vontade para compartilhar seus comentários ou dúvidas.

Iniciando uma jornada em observabilidade - O11y.

Nicolas Takashi — Mon, 17 May 2021 21:38:13 +0000

No início do ano, tive a oportunidade de fazer parte da equipe de observabilidade e sem dúvida, aceitei o desafio.

E hoje, vou começar algo diferente, a ideia principal é escrever algo como um diário de bordo descrevendo meus desafios durante meu trabalho como Engenheiro de Software em um projeto de observabilidade.

Eu me inspirei em dois de meus companheiros de equipe, schulzwill, que é o pioneiro nessa jornada ao meu lado, e el_luis_parada, que está fazendo a mesma coisa ao descrever sua jornada de estudo sobre o Site Reliability Engineer e você pode verificar este conteúdo clicando aqui.

Agora que você já sabe quem me inpirou e qual é o objetivo dos posts sobre O11y, vamos começar.

Mas primeiro, deixe-me dar um spoiler: não vou mostrar nada relacionado a código ou ferramentas hoje, só porque precisamos começar do início, entendendo o que significa observabilidade e alguns conceitos principais que eu precisei entender ao longo dessa jornada.

O que significa observabilidade?

Observabilidade é a nova criança no grupinho de palavras da moda que ronda a indústria de TI, e você verá muitas empresas usando isso para vender seus produtos e mostrar como são boas desenvolvendo software.

Mas a verdade é que a observabilidade não é nada novo. Como podemos ver nesta página da Wikipedia, a observabilidade vem da teoria de controle, com o propósito de medir o quão bem um sistema está funcionando com dados externos.

Agora você deve estar se perguntando algo assim:

O que significa observabilidade para um profissional de TI
Como tirar proveito da observabilidade
Como criar plataformas observáveis?

Bem, se isso te relaxar, eu fiz as mesmas perguntas quando ouvi sobre O11y pela primeira vez, então vamos responder essas perguntas.

Observabilidade para profissionais de TI

Todos os profissionais de TI precisam operar sistemas de produção para garantir que esses sistemas funcionem conforme o esperado. Quando esses sistemas não estão funcionando bem, devemos saber por que e como solucionar os problemas de maneira adequada.

Vamos usar o antigo exemplo de e-commerce e uma simples jornada do usuário para comprar sapatos novos em sua plataforma de e-commerce favorita.

Para simplificar, não irei cobrir todos os fluxos existentes relacionados a transporte, estoques, etc.

Olhando para o exemplo acima, podemos verificar se existem algumas interações entre o usuário e a plataforma de e-commerce.

Agora imagine a jornada do usuário em uma grande plataforma de e-commerce, quando eles estão enfrentando eventos significativos, como:

Black Friday
Singles Day
Saldos

Durante esses eventos, as interações do usuário podem criar centenas ou milhares de chamadas de rede com muitos fluxos de requisições diferentes.

Solucionar problemas de sistemas no cenário descrito sem observabilidade certamente será um trabalho árduo. Por isso, os profissionais de TI devem ter plataformas observáveis para operar esses sistemas com base em dados, procurando Métricas, Logs e Trace.

Como criar plataformas observáveis?

Bem, desculpe, mas não é um trabalho fácil, mas posso garantir que é um trabalho satisfatório.

Se você é uma pessoa atenta aos detalhes, viu que marquei três palavras em negrito no último tópico: Métricas, Logs e Trace, que são considerados os três pilares da observabilidade.

Olhando para esses três pilares, você pode estar pensando que é bastante simples ter plataformas observáveis, mas não se apresse em tirar conclusões tão rápido. Se você tiver métricas, logs e rastreios, isso não significa que haja capacidade de observação em seu sistema.

Mais do que ferramentas e padrões, a observabilidade é a capacidade de relacionar todas as informações coletadas para responder a perguntas como:

Por que “y” está quebrado?
O que deu errado durante o lançamento do recurso “x”?
Por que o desempenho do sistema diminuiu nos últimos meses?
Como estava meu serviço no ponto “x”?
Este problema de sistema está afetando usuários específicos ou todos eles?

Muitas ferramentas o ajudarão a coletar as informações adequadas sobre métricas, registros e rastreamentos, como:

DataDog - (Requer uma licença comercial)
Splunk - (requer uma licença comercial)
New Relic - (Requer uma licença comercial)
ElasticSearch - (gratuito e de código aberto)
Prometheus - (gratuito e de código aberto)
Jaeger - (gratuito e de código aberto)

Existe um oceano de possibilidades para ter sistemas observáveis, para que você possa construir sua plataforma O11y ou comprar uma solução de fornecedor.

Como tirar proveito da observabilidade

Começamos a tirar proveito de plataformas observáveis quando começamos a responder às questões mencionadas acima.

Os sistemas observáveis nos fornecerão dados que ajudarão as áreas de tecnologia e os negócios a tomar boas decisões, como:

Prever como dimensionar sua infraestrutura com base em eventos anteriores.
Descobrir qual é o caminho da solicitação dentro do seu sistema?
Criar alertas com base em comportamento inesperado

São apenas exemplos de coisas que podem ser melhoradas ou implementadas quando temos sistemas observáveis. Como podemos ver, tanto a área de tecnologia quanto a área de negócios podem tirar proveito disso e fornecer soluções genuínas aos seus clientes.

Conclusão

Observabilidade é de fato uma criança nova no grupinho, então todas as outras crianças querem brincar com a observabilidade, especialmente quando podem brincar com brinquedos como Kubernetes e Arquitetura de Microsserviços.

Como você pode imaginar, observabilidade é um tópico vasto e não posso resumir tudo em uma único blog post. Mas já temos os conhecimentos básicos para avançar para as seguintes disciplinas.

Espero que você goste.
Deixe-me saber o que você acha disso.
A observabilidade é um processo crucial para as empresas?

Designing Restful APIs using an API-First Approach — Contract Test

Nicolas Takashi — Thu, 29 Oct 2020 08:46:33 +0000

A few days ago I wrote the second part of the "Designing Restful APIs using an API-First approach" post series, I really recommend you check the second post to understand how OpenAPI will help you have a MockServer without writing any code.
Today we will talk about contract testing and how to ensure that the API implementation meets the design proposed.

When we talk about APIs, we are talking about contract definition, so we must have in mind that after an API is made available, the defined contract must be supported by the API provider.

Even after the API was made available, it keeps evolving, and one of the biggest challenges during an API Life-cycle is the API evolution. Because sometimes, business requirements change so much, that the current contract doesn’t meet new requirements and a breaking change is required.

😈 Breaking Changes — The root of all evil

As I usually say APIs are contracting definitions and we can’t break a contract without any bad consequence.

When a new functionality breaks the current state of the contract, if you don’t manage it very well, you will have problems with your customers that will have their clients broken after a new release of your API.

So in order to have fast feedback during the API development, you must know if a change introduces a breaking change.

🧪 Possible Approaches

There are a couple of possible approaches that can help you achieve fast feedback about breaking changes.

Unit testing

Running a unit test against your API will let you know when the contract is broken because your test will start to fail.

It's a very easy strategy to start, and almost the applications already have a unit test suit, in the order hand, this approach is too easy to be bypassed because the developer can just fix the test to succeed and move forward.

API Versioning

Create a new version of API for each new release, keeping old releases until your clients move to the new one.

We know that API Versioning is a best practice, but if you create a new version of your API, for each release, without a shadow of a doubt this will become messy very fast, and you will have big problems keeping all those versions together.

Contract Test

Check for differences between what was designed during the API Design and what was developed, running it in the Continuous Integration pipeline will provide us fast feedback, this is a very flexible and effective approach.

What approach should I use?

If we look at the first two options, we can find clear drawbacks, either by process bypass or maintainable problems.

Comparing the Contract Test against two other approaches, we can see that developers could not bypass the process without making a change in the original OpenAPI Document, this change must be reviewed for your team and if possible API Stake Holders as well.

It's easier to be automated and provide a clear understanding of what is falling and why it is failing.

🤖 Contract Test Implementation

As described above contract test seems to be the best solution to detect breaking changes in build time, and to help us during this journal, let's use a tool called openapi-diff, this is an npm package that compares two OpenApi documents and looks for what was removed or added.

OpenAPI-diff separates changes into two groups, Breaking Changes and Smooth Changes, let's see some examples:

Breaking Changes

Delete Path or Parameter
Rename a Path or Parameter
Add Required Parameter
Modify Response Item

Smooth Changes

Add Path or Parameter
Add Response Item
Add or Update Descriptions

To install openapi-diff is too easy, you just need to run one of the commands below.

// using npm
npm install openapi-diff

// using yarn
yarn add openapi-diff

Test Against What?

Probably you are asking yourself, but against what we will check by differences.

A very important step before we move forward in the next steps is to extract the swagger.json generated by your code, it will depend on the platform that you are using to write your application.

In this blog post, I will use an ASP.NET Core API that will be in the same repository that is being used for this series.

To keep it simples, I won’t show how to set up the Swashbuckle in an ASP.NET Core project, but if you want to see how to do that properly, comment below and I can write a post dedicated to this setup.

ASP.NET Core and Swashbuckle CLI

Since you have Swashbuckle configured into your ASP.NET Core project, you can install a Dotnet tool named Swashbuckle.AspNetCore.Cli, it's a very simple tool that extracts the swagger.json using the project DLL, if you want to know more about this tool, please check the Github Document.

After you install the tool as described in the official documentation, you just need to run the command below.

swagger tofile --yaml --output ./bin/swagger.yaml ./api/TodoApp.Api/bin/Debug/netcoreapp3.1/TodoApp.Api.dll v1

The command above will extract the swagger.json and convert it to a YAML file, this document looks like the OpenAPI document that we wrote during the API Design process.

OpenAPI-Diff in Action

Now that we have the OpenAPI Document that was designed and the OpenAPI Document that was generated through the code, it's time to compare those files and check the output.

Just run the command above and let see the console output.

openapi-diff ./bin/api.yaml ./bin/swagger.yaml

Without Changes

With Changes

I just made some changes in the code, add a new query string parameter to the GET /tasks endpoint, and removed the status code from the GET /tasks/{id} endpoint, we can see the output below.

As you can see above, the change in the query string parameter is not logged, but the change that removed the status code was detected as a breaking change.

Now you can just add the OpenAPI Diff to your API Pipeline to detect possible breaking changes every time that someone changes the API implementation and push it to your repository.

This strategy allows the developer to run the same script in their local machine avoiding broken pipelines, having a fail-fast validation.

🏁 Conclusion

In my opinion, avoid breaking changes is the most difficult task during the API Life Cycle, and in the majority of the time its related if the Design Session, if we take care of Design for longevity and try to understand possibles corner cases of the domain that the API aims to solve, we can reduce this necessity to break the API Contract when new business requirements appear.

In order to have breaking changes under your control, a contract test strategy may help us with fast feedback and keep us safe to break the contract and create problems for the clients of your API.

As usually every code will be updated in the Github Repository to you run to use it as a complement of this post.

Let me know what do you think about this strategy?
Do you already know it?

Comment below and let's share the experiences.
I hope you enjoy it, see you soon.

Designing Restful APIs using an API-First Approach — Mocking

Nicolas Takashi — Tue, 06 Oct 2020 08:29:52 +0000

A few weeks ago, I began a new post series to talk about API-First and API-Design using OpenAPI Specification.

If you didn't see the first post of this series yet, I recommend you stop reading and check it by clicking here.

In today’s post, I will address how to build a Mock Server to improve the developer experience and work parallelism, using OpenAPI documents.

😎 — Mock Concepts

Before we deep dive into Mock techniques using OpenAPI documents, let’s just ensure that everybody knows what Mock means according to Wikipedia.

In object-oriented programming, mock objects are simulated objects that mimic the behavior of real objects in controlled ways, most often as part of a software testing initiative. A programmer typically creates a mock object to test the behavior of some other object, in much the same way that a car designer uses a crash test dummy to simulate the dynamic behavior of a human in vehicle impacts. The technique is also applicable in generic programming.
Wikipedia

💡— Mock at the HTTP level.

In the API worlds, the concept of mock remains the same described above, but instead of using frameworks to mockup objects such as Moq and RinoMock, this is done using a Mock Server.

The Mock Server will respond to the expected endpoints, error for non-existent endpoints, often even provide realistic validation errors if a client sends it an invalid request.

Mock Server Alternatives

There are a couple of alternatives to create a Mock Server, each one with its trade-offs, below you can check a short options list.

Every option listed above are great tools and definitely works as expected. But the goal of this post is to show how to use the OpenAPI documents to build a mock server.

Bearing it in mind, just Prism provides us a built-in integration with Open-API to create a Mock Server, so we don’t need to write any code to do that.

🙌 — Introduction to Prism

Prism is a command-line interface that aggregates a set of packages for API Mocking using OpenAPI documents, this was developed using JavaScript and NodeJS, there are a huge community and many stars on Github Repository.

Getting Started

In the last post, I used a Github Repository with the code of the post series, so let’s keep working on top of that and evolves the solution.

Now let's add Prism to the project using some of the commands below.

yarn add @stoplight/prism-cli
# or
npm install --save @stoplight/prism-cli

Prism’s HTTP mock server simulates a real web API by providing endpoints and validation rules described in your API description document, like any HTTP solution, it works around the Request Messages and Response Messages.

Response Generation

Prism will try to return meaningful responses based on whatever information it has available, this means that any OpenAPI document can be used, but better documents provide better results. If you want to know how the prism decision engine works for response generation, you can check this link.

Response Generation Strategies

Prism has two response generation strategies, static generation, and dynamic generation, there are several differences between those options that we will understand right now.

Static Generation Strategy

By default, Prism will use a static generation strategy to create a response message, to use it, you can run the command below.

prism mock <path-to-openapi>

If the provided OpenAPI document has a response body example then it’ll use that, otherwise, the response body will be created by looking through the whole schema object to create a fake response.

Dynamic Generation Strategy

Testing against the exact same piece of data over and over again is not the best way to build a robust API, in the real world the data is dynamic and we must be able to handle it properly, you can run the command below.

prism mock <path-to-openapi> -d

Dynamic mode solves this by generating a random value for all the properties according to their type, and other information like format, this means that the more descriptive your API is, the better job Prism can do at creating a mock response.

Request Validation

Based on the API description document Prism can take into consideration all sorts of validation rules for the request body, headers, query parameters, using keywords like type, format, maxLength.

👊 — Introduction to Prism

Now you know the Prism basics and how it works, let's configure it in our API project and start a new mock server.
First of all, we need to change the package.json to looks like the example below, with two new commands mock and premock.

{
  "name": "todoapi-spec",
  "version": "1.0.0",
  "description": "OpenAPI specification for TodoAPI",
  "main": "index.js",
  "author": "Nicolas Takashi",
  "license": "MIT",
  "scripts": {
    "premock": "yarn bundle",
    "mock": "prism mock ./bin/api.yaml -d",
    "bundle": "swagger-cli bundle specs/api.yaml -o ./bin/api.yaml -t yaml",
    "prelint": "yarn bundle",
    "lint": "spectral lint ./bin/api.yaml --ignore-unknown-format"
  },
  "dependencies": {
    "@openapitools/openapi-generator-cli": "^1.0.15-4.3.1",
    "@stoplight/prism-cli": "^4.0.0",
    "@stoplight/spectral": "^5.5.0",
    "swagger-cli": "^4.0.4"
  }
}

Now, just run the command below to start the Prism, and look to the console to see the output.

yarn mock

If we look to the console output, there is a Mock Server running through the localhost:4010, and from now, we are able to make API calls to this Prism Server.

Making an API Call with Swagger UI

Let's make a request call through the Swagger UI, but before you do that, change the api.yaml and add the servers section, like the example below.

openapi: 3.0.0
info:
  title: Todo App API
  description: Todo App API.
  version: v1
  contact:
    email: 'nicolas.tcs@hotmail.com'
    name: 'Nicolas Takashi'
servers:
  - url: http://127.0.0.1:4010
    description: Mock Server
tags:
  - name: Tasks
paths:
  '/tasks':
    $ref: './components/paths/tasks/task.yaml'
  '/tasks/{id}':
    $ref: './components/paths/tasks/task-by-id.yaml'
components:
  schemas:
    Task:
      $ref: 'components/schemas/task.yaml#/components/schemas/Task'
    PagedTask:
      $ref: 'components/schemas/pagedTask.yaml#/components/schemas/PagedTask'
    ProblemDetails:
      $ref: 'components/schemas/problemDetails.yaml#/components/schemas/ProblemDetails'

Now if we look to the Swagger UI, we will notice a new drop-down with a list of available servers, such as the image below.

Last but not least, let’s try to create a task, calling the POST /tasks and looks at the output.

Success Request

Since the name is required and we sent this property, the request returns an HTTP Created 201 with the location header, as you can see in the image below.

Invalid Request

After tried to create a task without a name, we get back the HTTP Bad Request, as described in the OpenAPI document.

🏁Conclusion

That’s all for today, now you have the basics about the Mock APIs, I really recommend you to check the Prism official documentation, to understand every feature and how to deploy it to your clients take advantage of this strategy.

API Mock can anticipate so many problems that could exist between APIs consumption and system integrations.

You can check the project on Github, everything is updated with the implementations that we did in this post.

I hope you enjoyed it, please let me know your feedback, comment below, and share with your friends.

Designing Restful APIs using an API-First Approach

Nicolas Takashi — Wed, 30 Sep 2020 11:38:28 +0000

I’ve been working with RESTFul APIs for some time now and one thing that I love to do is to talk about APIs.
So, today I will show you how to build an API using the API-First approach and Design First with OpenApi Specification.

First thing first, if you don’t know what's an API-First approach means, it would be nice you stop reading this and check the blog post that I wrote to the Farfetchs blog where I explain everything that you need to know to start an API using API-First.

📋 — Preparing the ground

Before you get your hands dirty, let’s prepare the ground and understand the use case that will be developed.

Tools

If you desire to reproduce the examples that will be shown here, you will need some of those items below.

NodeJS
OpenAPI Specification
Text Editor (I’ll use VSCode)
Command Line

Use Case

To keep easy to understand, let’s use the Todo List App, it is a very common concept beyond the software development community.

Disclaimer

I will hide some information to keep easy to read from the gist files, but at the end of this post, I will share the Github repository link, so you can access the full content.

🙌 — Get your hands dirty

Now that you already have your environment with the needed dependencies, let’s have some fun and start to see the things happening.

The first step is to create the entry point file for the OpenAPI specification and add some content to this file.

openapi: 3.0.0
info:
  title: Todo App API
  description: Todo App API.
  version: v1
  contact:
    email: 'nicolas.tcs@hotmail.com'
    name: 'Nicolas Takashi'
paths:
#... there more content here.

The file above is a very simple representation of an OpenAPI document, just to introduce you to the concept.

OpenAPI is a self-explanatory notation, but if you don’t know how this works yet, I recommend to use the Oficial documentation to support you during the spec design process.

Writing Path Objects

The Path Object holds the relative paths to each endpoint and their operations, let's add a file named tasks.yaml that should look like that.

get:
  operationId: get-tasks
  summary: Returns a paged list of tasks.
  description: Returns a paged list of active tasks
  tags:
    - Tasks
  parameters:
    - $ref: '../../parameters/page-parameters.yaml#/components/parameters/page'
    - $ref: '../../parameters/page-parameters.yaml#/components/parameters/pageSize'
  responses:
    '200':
      description: Success
      content:
        application/json:
          schema:
            $ref: '../../schemas/pagedTask.yaml#/components/schemas/PagedTask'
post:
  operationId: create-task
  summary: Create a task
  description: Create a new task
  tags:
    - Tasks
  requestBody:
    required: true
    content:
      application/json:
        schema:
          $ref: '../../schemas/task.yaml#/components/schemas/Task'
  responses:
    '201':
      description: Created
      headers:
        Location:
          description: 'Location of the created task /tasks/{id}'
          schema:
            type: string

Above you can see that was added two operations, that will enable the Todo List API to create a task and get a paged list of tasks.

Now we can add the operation to enable the API to remove a task, find one specific task, and update a task, to do that we must create a new file named tasks-with-id.yaml and add the following content.

get:
  operationId: get-task
  summary: Get task by id
  description: Return a task
  tags:
    - Tasks
  parameters:
    - in: path
      name: id
      description: Task id
      required: true
      schema:
        type: string
        format: uuid
  responses:
    '200':
      description: Success
      content:
        application/json:
          schema:
            $ref: '../../schemas/task.yaml#/components/schemas/Task'
put:
  operationId: update-task
  summary: Update a task
  description: Update a task
  tags:
    - Tasks
  parameters:
    - in: path
      name: id
      description: Task id
      required: true
      schema:
        type: string
        format: uuid
  requestBody:
    required: true
    content:
      application/json:
        schema:
          $ref: '../../schemas/task.yaml#/components/schemas/Task'
  responses:
    '202':
      description: No Content
delete:
  operationId: delete-task
  summary: Delete a task
  description: Delete a task
  tags:
    - Tasks
  parameters:
    - in: path
      name: id
      description: Task id
      required: true
      schema:
        type: string
        format: uuid
  responses:
    '202':
      description: No content

Probably you can understand almost everything as described in the file above, but I believe the $ref notation deserves some explanation.

Using references

In software development is very common to share code, and, the same happens when we are designing API. Bering in mind the OpenAPI specification able us to re-utilize common objects across an OpenAPI document.

Using the notation $ref you can add local or remote references to the API, as you can see in the examples above.

If you want to understand better how to work with references and where you can use it, please take a look at the official documentation.

Writing Schema Objects

We are using a schema component named Task, this schema is used across the API to represent the resource tasks, this schema looks like the example below.

components:
  schemas:
    Task:
      type: object
      required:
        - name
        - id
      properties:
        id:
          description: Task unique identifier
          type: string
          format: uuid
          readOnly: true
        name:
          description: Task name
          type: string
          example: Todo Service

With the code above we can re-utilize the Task schema across all the API and avoid code duplication.

Writing Parameter Object

Just like the schemas, we can do the same with the parameters, if there are parameters that are shared across the API, we can reference a local or remote file, here we will create two parameters to represent the information about the page and its size that must be like the following example.

components:
  parameters:
    page:
      in: query
      description: Desired page
      name: page
      schema:
        type: integer
        format: int32
        default: 1
        minimum: 1
    pageSize:
      in: query
      description: Desired quantity of itens per page
      name: pageSize
      schema:
        type: integer
        format: int32
        default: 10
        minimum: 10
        maximum: 100

The final API entry point

After a lot of separated examples of files, let’s see how everything looks together.

openapi: 3.0.0
info:
  title: Todo App API
  description: Todo App API.
  version: v1
  contact:
    email: 'nicolas.tcs@hotmail.com'
    name: 'Nicolas Takashi'
tags:
  - name: Tasks
paths:
  '/tasks':
    $ref: './components/paths/tasks/task.yaml'
  '/tasks/{id}':
    $ref: './components/paths/tasks/task-by-id.yaml'
components:
  schemas:
    Task:
      $ref: 'components/schemas/task.yaml#/components/schemas/Task'
    PagedTask:
      $ref: 'components/schemas/pagedTask.yaml#/components/schemas/PagedTask'

Everything looks good, we have a Todo List API designed, following with a good design, right now if you are using Visual Studio code like me, you can install an extension named Swagger View, so you can render the specification such example below.

As you can see the swagger view looks great and every client can explore your API through the Swagger-UI.

📦 Bundling OpenAPI Documents

Before start to interact with the OpenAPI documents to lint, create mock servers and SDKs for our clients, we must bundle the API to enable machine read and interact with that.

The concept of splitting the OpenAPI documents into many files and reference those files later is an approach to help you during the design process, avoiding duplications and enhance code reuse.

But it's not a valid OpenAPI document and right now, we'll see how to bundle many documents into one using swagger-cli.
To install the swagger-cli is very simple, just run the command above and it's done.

// using npm
npm install swagger-cli

// using yarn
yarn add swagger-cli

After that, you just need to run the bundle command below.

swagger-cli bundle api.yaml -o ./bin/api.yaml -t yaml

Done, you already have the API bundled into one file only, now you are able to lint this file ensuring that every guideline was used.

📐 - RESTFul Guidelines

A very important thing that must be done, when we are designing APIs is to define guidelines.

There are a bunch of guidelines to design RESTFul APIs provided by the software community, but in many cases, those guidelines don’t meet your business requirements and you need to create your own guidelines.

Much more important than define guidelines is to ensure that your APIs are following these guidelines, for instance, ensure that every request has a 200 response.

But we know that it’s very difficult to do this job without a tool to help us, so bearing it in mind let me introduce you to a good friend to support you during this job.

Using spectral to validate OpenAPI documents

Check if your APIs are being designed following the proposed guidelines by your company, which could be a difficult job to do manually.

Thereby we can ask for a tool like Spectral to help us and create an autonomous process, to do this job.

Spectral is an open-source project that works like a linter to OpenAPI documents and has a lot of rules to ensure very common guidelines in the software industry to develop RESTFul APIs.

To use spectral is very simple, you just need to install the node package using the command below.

// using npm
npm install @stoplight/spectral

// using yarn
yarn add @stoplight/spectral

Before executing the command to lint and check if the API meets the guidelines, you need to bundle the API as mentioned before, after that just run the command below.

spectral lint ./bin/api.yaml --ignore-unknown-format

After that, you can see the output informing if there are errors or not, such as the image below.

Spectral success output

Spectral error output

If you want to understand how the spectral works, what are the options available to lint, and how to create your own rules or ignore existing rules, I really recommend you visit the official documentation, they have awesome examples of how to do that.

In addition to the main benefit that spectral gives us, we can also take advantage and add this in the process of continuous integration or even via git hook, ensuring that we will always have the OpenAPI documents validated and compliant with the guidelines.

🏁Conclusion

Let’s stop here in this post, I really want to talk more about API-First and OpenAPI but to a first post, there is a lot of information that must be digested.

The idea behind this post is just to show how to start designing an API with an API-First approach and OpenAPI Specification. In the next post, we will talk about Mock Servers and SDKs, how we could take advantage of OpenAPI to help improve the development process.

I hope you enjoyed the content and please if you have any doubts or suggestions let me know, write a comment, or hit me on twitter or somewhere else.

Github Repository with Examples