List of Topics:
Location Research Breakthrough Possible @S-Logix pro@slogix.in

Office Address

Social List

QoS-Aware Serverless Workflow Scheduling in AWS Lambda

QoS-Aware

QoS-Aware Serverless Workflow Scheduling in AWS

  • Use Case: Modern applications (e.g., real-time video processing, fraud detection, IoT data analytics) often consist of complex workflows—a sequence of dependent tasks.Traditionally, workflows run on VMs or containers, but serverless computing (AWS Lambda) offers on-demand execution, scalability, and cost-efficiency.

Objective

  • Develop a QoS-aware scheduling framework for workflows in AWS Lambda.

    Optimize for deadline, execution cost, and reliability.

    Compare different strategies:

    Heuristic-based (e.g., Min-Min, Deadline-Met Heuristic).

    ML-based (e.g., Reinforcement Learning, Graph Neural Networks).

    Measure impact on latency, throughput, fault tolerance, and cost under varying workloads.

Project Description

  • Simulate and deploy workflow-based applications (like ETL pipelines, IoT event processing, genome sequencing workflows) using AWS Lambda and Step Functions.

    Model workflows as Directed Acyclic Graphs (DAGs) where nodes = tasks and edges = dependencies.

    Placement: Decide which Lambda function executes which task.

    Ordering: Determine when tasks should execute to meet deadlines.

    Scaling: Decide whether to allocate more parallel executions.
  • Two approaches will be implemented and evaluated :

    Heuristic-based Scheduling → Minimize makespan or cost using rules.

    ML-based Scheduling → Use historical logs and reinforcement learning to predict task placement decisions.
  • AWS Services & Purpose :
    Service Purpose
    AWS Lambda Executes serverless functions (workflow tasks) without managing servers.
    AWS Step Functions Orchestrates workflows as a DAG of Lambda functions, handling task dependencies.
    Amazon CloudWatch Monitors latency, errors, cost, and execution logs for real-time metrics.
    Amazon DynamoDB Stores workflow metadata, task execution times, and ML training data for scheduling.