API CodeFlow: An Agile, Design-First Workflow for API Development

Ted Epstein
Posted by
Ted Epstein on Oct 02, 2018 04:17 PM
API CodeFlow Blog Image.png

When RepreZen joined the OpenAPI Initiative last year, I waxed poetic in a guest post about OpenAPI as a REST API description language:

"The OpenAPI Specification is a beautiful example of what just the right amount of abstraction can do. It offers a step up in expressive power, allowing developers to think in terms of resources, operations and data models — familiar concepts, gliding at a comfortable altitude just above the canopy of protocols, controllers and classes.  And you don’t have to rethink your API in high-level semantics, esoteric concepts, or religious dogma."

Many developers still think of OpenAPI, with its roots in the Swagger ecosystem, as an API documentation language. But in our client engagements, we've had remarkable success in using OpenAPI and RAPID‑ML -- our own domain-driven API modeling language -- as a first-class foundation for API development

The RepreZen team decided to take the best practices, examples, and success factors from these projects, and package them together into a process pattern that we call API CodeFlow. By adhering to a few essential guidelines defined in API CodeFlow, we're able to execute agile projects with consistently faster cycle time, higher velocity, and better API design quality as reported by API client developers.

Code-First or Contract-First?

API CodeFlow combines two paradigms that are often seen at opposite ends of the SDLC spectrum:

  • API CodeFlow is Agile and code-focused. It promotes a responsive customer feedback loop, using an iterative, evolutionary development process. And it respects the need for developers to own and control their source code.
  • API CodeFlow is Specification-Driven. Changes to the API design start in the OpenAPI document or RAPID-ML specification, and the code flows downward from there. The API specification and implementation proceed in lockstep through the development lifecycle. 

API-CodeFlow-InfoGraphic-Center-Shadow

We usually hear about code-first or contract-first as alternative strategies to keep the API implementation and documentation in sync.

Code-first uses specialized annotations in the API implementation codebase, and extracts the API specification and documentation from the annotated code at runtime or compile-time. This has obvious appeal for developers who don't want to context-switch into "API design" as a separate exercise, with its own separate environment, specification format, cadence, and governance process. Code-first API development works well for small projects, and it's "agile" at least in the narrow sense of prioritizing developer flow and working software over a fully elaborated design.

Contract-first -- sometimes called specification-first, API design-first, or simply API-first -- uses OpenAPI or another API description format in a conscious, collaborative design process. The team evolves the API design through an iterative process, soliciting feedback from API client developers, and optimizing alignment with client developers' requirements and domain perspective. Teams may use code generation to create a scaffold for implementation, but usually only once, or infrequently in response to major API design changes. So there's an association of contract-first workflows and code generators with waterfall lifecycle models.

So where does API CodeFlow stand? Is it code-first, or contract-first?

The Contract Is the Code

API CodeFlow 1 - DesignAPI CodeFlow is explicitly design-first, but it regards the API design document as just a specialized form of code. And it treats the code generator as a specialized compiler for OpenAPI, RAPID-ML, or another machine-readable API description format. 

With API CodeFlow, you manage the API contract as a first-class part of the codebase, and your team can collaborate on the API design through the same version control, branching model, code review and integration processes you're already using. When the API contract is the code, the distinction between code-first and contract-first evaporates. And as we've found, you get the benefits of both, in good measure.

The API Specification Compiler

API CodeFlow 2 - Generate

Regrettably, PowerPoint Slides still don't compile. But machine-readable API specifications, in a manner of speaking, do.

Code generators like Swagger-CodegenOpenAPI-Generator and NSwag, all supported in RepreZen API Studio, transform your OpenAPI specifications into executable code.  The generated code is best thought of as a scaffold, more than an API implementation. But the generated code should be extensible, error-free, idiomatic to the target language and framework, and a whole lot faster to generate than to write by hand. 

API CodeFlow integrates code generation directly into the automated build script for your project. If your generated code is out of sync with the API specification or with your hand-written implementation code, you'll know right away, on the first build or automated test run.

Extensibility: Good Fences, Good Neighbors

API CodeFlow 3 - Implement

A word about extensibility:  The generated scaffolding code is just the bare bones of an actual API implementation. The code generator doesn't know about your business logic, your data persistence scheme, or other specifics of your service. You'll need to plug your own hand-written implementation code into into the generated scaffold.

The mechanisms for integrating your custom code will vary according to the target language and framework, and the code generator. Your code might use subclassing, interface implementations, injection, partial classes, or other extension mechanisms.

But this is key:

A well-designed code generator will generate extensible code, so you can complete your implementation without touching the generated code. 

The generated code should allow you to use at least one convenient extension mechanism that can be defined entirely in your own files, separate from the generated code. Separate folders, even better. More than one supported extension mechanism, better still.

Some code generators will create special generate-once files you're expected to modify and maintain; once the file exists, the generator won't overwrite them.

But aside from those special cases, you need to keep your hand-written implementation code separate from the generated scaffold, so it's always safe to regenerate. That's how API CodeFlow supports an iterative, evolutionary process. Your hand-written implementation code depends on the generated code, and in strongly typed languages this will expose breaking changes right away. This kind of enforced consistency is what we like to see. 

We've created open source example projects for Spring Boot and Node.js, demonstrating recommended project layouts and extension patterns for API CodeFlow.  Each language, framework, and generator requires a bit of experimentation to get API CodeFlow working smoothly. So we'll be adding more of these to the API CodeFlow resource page. And hey, your own contributions are always welcome!

Here's what the overall API CodeFlow process looks like: 

API CodeFlow

Code Generators Are Code Too

Sometimes you don't find exactly what you're looking for in an open source library. This can be especially important with code generation, where you might need the generated code to use a specialized set of libraries, target a specific framework version, or just adhere to a strict coding standard. 

RepreZen GenFlow Architecture

There's good news: Most open source code generators are reasonably simple and easy to adapt. They're template-driven, and the output template gives you some immediate orientation to find the part you need to modify.

And there's more good news: RepreZen API Studio allows you to write your own code generators using the Swagger-Codegen framework, or using RepreZen's built-in GenTemplate framework.  The API CodeFlow Node.js example demonstrates this, and you'll find complete documentation for custom code generation here.

"Sssshhh!  Don't tell them they're already doing model-driven development!"

At RepreZen, we make use of model-oriented design and development methods wherever we think they make sense. OpenAPI is a domain-specific modeling language for REST APIs. RAPID‑ML is another one. And RAPID-ML is built using the Eclipse Modeling Framework and Xtext, literally a DSL for DSLs. 

We've worked with some organizations who embrace model-driven development and domain-specific languages. But we also talk to a lot of developers who are highly skeptical and turned off by the idea of model-driven development. To those developers, MDD smells like pie-in-the-sky idealism, white lab coats, Gantt charts, giant UML diagrams, oppressive process maturity initiatives, and waterfall projects that never get done. 

(BTW, if Stephen Colbert is applepious, can I say MDD smells like apple-pie-in-the-sky?)

Sensing this hostility towards model-oriented development, we've mostly responded by shutting our apple-pie-holes. After all, if "model-driven" has negative connotations, we don't need to call it that. 

But what we're advocating with API CodeFlow is clearly model-driven. In fact it's a classic use of an external DSL as a platform-independent model (PIM), translated to a platform-specific model (PSM) via codegen.

The reality is that models, DSLs, and code generators are everywhere, whether or not we choose to use that terminology. Maven POMs, Dockerfiles, Avro schemas, and npm package.json files are all DSLs, just to name a very few. Adobe Illustrator generates Postscript and SVG. C++ started as a preprocessor for C.

So let's drop the artificial distinctions between model-driven development and... well, everything else.  If you're not sure whether API CodeFlow will work for you, I'd encourage you to try it, and tell us what you think!

Topics: API CodeFlow, Code Generation, Swagger Codegen, API Development, API Design, OpenAPI, Best Practices, SDLC

Welcome!

At RepreZen, we're building our business on two things:  thought leadership in API design, and great conversations.

This blog is one of many places where we'll have illuminating, mutually enriching conversations with our customers, partners, and the software community at large.  Please chime in with your thoughts and let's get started!

RepreZen API Studio - Free Trial!


Installing RepreZen API Studio is easy!  Here's how it works:

number_circle_1   Submit the trial registration form.
number_circle_2   Check your inbox for download instructions.
number_circle_3   Download and run the automated installer.

 

System Requirements

 

RepreZen API Studio runs on the following operating systems:

  • 64-bit Windows 7, 8.0 and 8.1
  • Mac OS X 10.7 or higher
RepreZen API Studio requires 64-bit Java 8 or higher.