The Traps of Frameworks

When I interview a Java developer, if I see Spring Boot or Spring on the candidate’s resume, I may start with a simple question: “What is the default scope for a Spring bean”? Most people would get it right. I would then follow with a tricky question: “Does Spring make sure a Singleton bean thread-safe?” or “Does developer need to do anything to make sure a Singleton bean thread-safe?”.

When I say “tricky”, not because it’s tricky technically, but because half interviewees have no idea. The other half who correctly answered don’t always demonstrate solid understanding of Singleton and thread-safety. It’s okay to guess at an interview I guess.

Spring Boot is one of those popular frameworks for Java developers. Like most other Java frameworks, it provides proven reusable libraries and increases productivity. Some developers can probably make a living by simply being good at it.

However, because it encapsulates the interpretation of various Java Specifications, hides the complexity of design and implementation, often the framework itself imposes a serious impediment for developers to understand the underlining fundamentals.

Many Spring Boot developers don’t know Spring Boot is just a framework on top of another popular framework Spring Framework, which was initially a framework for Java Servlet applications. Most freshly-minted Spring Boot developers never heard of Servlet, not to mention web.xml. They only know their Spring Boot applications, “just run”. They never know why and how it runs.

Because of that, they never think of what the underlining Servlet Container is, what the default configurations (like Max Concurrent Requests) are, and how to fine tune those configurations. Imagine asking them to write a Java Web Application without Spring Boot?

Frameworks tend to wrap a lot of default features and behaviors under the hood, just to name a few: default Encryption Algorithm, default Socket Timeout, default Retry Strategy.

In the past, Frameworks might have configuration property for each “feature”, but this has changed in the recent years. Nowadays, Framework authors tend to favor “Convention over configuration”. Old configuration files are replaced by annotations with “sensible defaults”. Moreover, many of the features and behaviors are “discovered” automatically based on your running environments, like system properties, environment variables and what is in the class-path.

Several years back, I led a framework team. We built a Framework as the foundation for a slew of web applications that support multi-million $ business. We worked very hard to support all major features by default, and still allow each application to extend and override each feature by configuration and automatic discovery. I learned first hand, it’s even harder for application developers to fully understand how each feature worked and how to extend or override them.

Naturally, due to the lack of visibility and transparency of frameworks, people makes a lot of assumptions about frameworks, such as Singleton bean thread-safety. Some of the assumptions will definitely haunt the team down the road if the technical leads on the team didn’t review the design and code carefully.

Overtime, frameworks will evolve or die. If you ever worked with Struts 1.x framework, and if you didn’t understand Java Servlet, you would have a difficult time to migrate your applications to Struts 2.x or Spring.

Frameworks are your tools, not your crutches. If you don’t think out of the box of Spring Boot, you can’t professionally outgrow Spring Boot. Simple. Period.

That is true to other frameworks too.

Frameworks can help you get started quickly, but understanding the underlining principles will help you in the long run.

Generic Asynchronous Retry Architecture

Murphy’s law says:

Whatever can go wrong, will go wrong.

It’s so true in software engineering.

To build resilient software applications, when architecting the integration points with downstream services, we shall consider all error scenarios. Robust error handling is essential. Retrying remote API calls is an important part.

A retry can be done either synchronously or asynchronously. If the clients require a response of the execution status, not just the acknowledgement of the receipt of the request, it’s appropriate to implement synchronous retries with limits on total retry numbers and time. On the other hand, if the clients don’t care about the actual execution status, or have ways to receive responses asynchronously, it is almost always a good idea to adopt asynchronous retry architecture. Of course, before putting a request into asynchronous retry process, we can always implement synchronous retry first whenever it makes sense.

In this article, we will focus on the asynchronous retry architecture.

2. Queuing for Asynchronous Retry Architecture

Queuing mechanism is the center of the Asynchronous Retry Architecture.

The originating service constructs a Retry Message that includes the original request info, the destination URL and other metadata, puts the Retry Message into an Async Retry Queue based on the chosen queuing system. A trigger could be configured in the queue to trigger a processor. Or, an Async Retry Processor can pull the queuing system for new messages. The Async Retry Processor can then utilize the message received from the Async Retry Queue and make another call to the destination downstream service.

A Dead Letter Queue is used to hold Retry Messages for certain period time after a (configurable) maximum number of retries have been reached.

The below figure is a very high level workflow and message flow:

3. Asynchronous Retry Architecture Diagram

Asynchronous Retry Architecture

In the above diagram, Service A is the calling service and Service B is the destination downstream service. If the initial call in Step 1 fails, Service A will put a Retry Message into the Async Retry Queue.

Depends on what Queuing System is chosen, either a trigger can be configured in the Async Retry Queue to trigger the Processor (3.1), or a Processor can be configured to poll the Async Retry Queue (3.2). If AWS SQS is chosen as the queuing system, a Lambda function can be configured to trigger the processor when a new message arrives.

Once the Async Retry Processor receives the Retry Message, it can use the request info in the message to reconstruct the request, and send the request to the destination URL that is also included in the retry message.

A Retry Message will be moved to the Dead Letter Queue if the maximum retry attempts have been reached as detected by the Async Retry Processor or the Async Retry Queue.

4. Generic Data Model for Retry Message

A Retry Message can have a generic data model as below:

<span id="60ab" class="hc wp tx so wq b gg wr ws x wt" data-selectable-paragraph="">{
    "request":{
        "url":"http[s]://$host:$port/$destitnation_endpoint_including_query_parameters",
        "method":"GET|POST|PUT|DELETE|PATCH",         
        "payload":"$json_string",
        "headers":[$headers_to_pass_to_the_target_service]
    },
    "receivedCount": "$number",
    "async-retry-queue":"$async-retry-queue[optional]",
    "dead-letter-queue":"$dead-letter-queue[optional]"
}</span>

With this generic data model design for Retry Message, an Async Retry Processor can be designed to process any Retry Message constructed by any originating services (Service A) to any destination services (Service B).

5. Retryable Errors

Only non-functional errors are retryable. Below are some examples:

a. No response at all;

b. Temporary Network issue, usually 5xx (http status) errors;

c. Request timeout: http status 408 errors;

d. Conflict: http status 409 errors;

e. Too many request: http status 429

f. If there is a Retry-After header in the http response of the downstream service;

h. Unauthorized: http status 401 errors with expired token error code/message. These kind of errors usually require a new token. In this case, the Async Retry Processor is responsible for getting the proper token.

6. Conclusion

Asynchronous Retry Architecture can be used to handle all retryable errors when the client is not expecting the execution result in the response of the call. It is extremely useful if a function may need to be tried many times for a long period of time.

The number of maximum retry attempts, the async retry queue name/url, and the dead letter queue name/url can all be configurable. The configurable values can make architecture flexible for many different applications.