Back

May 24, 2022

Maintaining Workflow Statuses For An Event-Driven Microservice Architecture

Overview

  1. The Event-Driven Microservice Architecture
  2. Simple (but not ideal) solution
  3. The Proposed Solution: Data Model For Statuses

The Event-Driven Microservice Architecture

Imagine that we are building a job application system using a set of microservices. This system needs to process job applications based on a defined workflow (i.e. a sequence/set of tasks) to completion.

For example, the set of tasks for a job application workflow may look like this:

  1. Application Form (Draft, Submitted)
  2. Phone Screen (Pending, Passed, Failed)
  3. On-site Interview (Pending, Passed, Failed)
  4. Background Check (Pending, Passed, Failed)
  5. Health Screening (Pending, Passed, Failed)
  6. Offer (Pending, Accepted, Rejected, No Offer, Rescinded)

In addition: #4 and #5 can be done in parallel.

Given the set of tasks, we may then have a set of microservices like this:

architecture

  1. ApplicationService: Processes applications and notifies PhoneScreenService when initiated by HR
  2. PhoneScreenService: Schedules phone screen sessions and notifies OnSiteInterviewService if passed
  3. OnSiteInterviewService: Schedules on-site interviews and notifies BackgroundCheckService and HealthScreeningService if passed
  4. BackgroundCheckService: Performs background checks on applicant
  5. HealthScreeningService: Initiates health screening session for applicant
  6. OfferService: Initiates offer to applicant after both background check and health screening are passed

Simple (but not ideal) solution

Perhaps a simple solution is to maintain a STATUS column in the APPLICATION_DETAIL table itself. This column would show the status such as ‘Submission’, ‘Phone Screen’, ‘On-site Interview’, etc…

CREATE TABLE APPLICATION_DETAIL (
  ID                INT PRIMARY KEY NOT NULL,
  FIRST_NAME        CHAR(100),
  LAST_NAME         CHAR(100),
  POSITION          CHAR(100),
  STATUS            CHAR(50)
);
ID FIRST_NAME LAST_NAME POSITION STATUS
1 PETER PARKER SOFTWARE ENGINEER SUBMISSION
2 MARY JANE RECRUITER PHONE_SCREEN
3 HARRY OSBORN SITE RELIABILITY ENGINEER ON_SITE_INTERVIEW
4 GWEN STACY ENGINERING MANAGER OFFER

This solution may work alright if:

  1. Each task do not need sub-statuses (e.g. Pending, Passed, Failed)
  2. Each task is executed in sequence (i.e. only one task is pending at any given time)

We quickly observe that this would not work well for our job application system given that:

  1. Each task has multiple sub-statuses (e.g. Pending, Passed, Failed)
  2. A single status cannot show
    • when multiple tasks are pending in parallel
    • a status overview of all the microservices

The Proposed Solution: Data Model For Statuses

Use a STATUS table to maintain the statuses of all your microservices:

CREATE TABLE APPLICATION_STATUS (
  ID                            INT FOREIGN KEY NOT NULL,
  PHONE_SCREEN_STATUS           CHAR(50),
  ON_SITE_INTERVIEW_STATUS      CHAR(50),
  BACKGROUND_CHECK_STATUS       CHAR(50),
  HEALTH_SCREENING_STATUS       CHAR(50),
  OFFER_STATUS                  CHAR(50)
);
ID APPLICATION_SERVICE PHONE_SCREEN_STATUS ON_SITE_INTERVIEW_STATUS BACKGROUND_CHECK_STATUS HEALTH_SCREENING_STATUS OFFER_STATUS
1 DRAFT          
2 SUBMITTED PENDING        
3 SUBMITTED PASSED PASSED PENDING PENDING  
4 SUBMITTED PASSED PASSED PENDING PENDING  
5 SUBMITTED PASSED PASSED FAILED CANCELLED NO_OFFER
6 SUBMITTED PASSED PASSED PASSED PASSED ACCEPTED

Pros:

  1. This table provides a good status overview of each application.
    • 1 is in DRAFT status
    • 5 FAILED the Background Check, and hence Health Screening is CANCELLED
    • 6 PASSED all stages and the offer was ACCEPTED
  2. The PENDING status is indicative of which microservice is holding up the application
    • 2 is PENDING on Phone Screen and nothing else (as denoted by the blank statuses)
  3. Querying this table by SQL is easy and hence comparing statuses across multiple applications is easy as well

Cons:

  1. When we were to use this single status table to handle different types of workflows, we end up with many empty cells when the task is not applicable to the workflow.