Reactive by Design
Built on Mutiny for non-blocking, high-performance applications
The Pipeline FrameworkReactive Pipeline Processing
Build scalable, resilient pipeline applications with Quarkus and Mutiny
Skip to content
Visual Pipeline Designer Available
The Pipeline Framework includes a visual canvas designer at https://app.pipelineframework.org that allows you to create and configure your pipelines using an intuitive drag-and-drop interface. Simply design your pipeline visually, click "Download Application", and you'll get a complete ZIP file with all the generated source code - no command-line tools needed!
Introduction
The Pipeline Framework is a powerful tool for building reactive pipeline processing systems. It simplifies the development of distributed systems by providing a consistent way to create, configure, and deploy pipeline steps.
Key Features
- Reactive Programming: Built on top of Mutiny for non-blocking operations
- Immutable Architecture: No database updates during pipeline execution - only appends/preserves, ensuring data integrity
- Annotation-Based Configuration: Simplifies adapter generation with
@PipelineStep - gRPC & REST Flexibility: Automatic adapter generation for fast gRPC or easy REST integration
- Multiple Processing Patterns: OneToOne, OneToMany, ManyToOne, ManyToMany, SideEffect and blocking variants
- Health Monitoring: Built-in health check capabilities
- Multiple Persistence Models: Choose from reactive or virtual thread-based persistence
- Modular Design: Clear separation between runtime and deployment components
- Auto-Generation: Generates necessary infrastructure at build time
- Observability: Built-in metrics, tracing, and logging support
- Error Handling: Comprehensive error handling with DLQ support
- Backpressure Management: Reactive processing with configurable backpressure strategies
Getting Started
New to The Pipeline Framework? Start with our Quick Start guide to learn the basics using the visual Canvas designer.
Guides
To get started with The Pipeline Framework, explore these guides:
Getting Started
- Getting Started: Setting up the framework in your project
- Creating Pipeline Steps: Building your first pipeline steps
Application Development
- Application Structure: Structuring pipeline applications
- Backend Services: Creating backend services that implement pipeline steps
- Orchestrator Services: Building orchestrator services that coordinate pipelines
Advanced Topics
- Pipeline Compilation: Understanding how the annotation processor works
- Error Handling & DLQ: Managing errors and dead letter queues
- Observability: Monitoring and observing pipeline applications
Reference
- Architecture: Deep dive into the framework architecture
- Framework Overview: Complete architecture and comparison to original spec
- Reference Implementation: Complete implementation guide with examples
- YAML Configuration Schema: Complete YAML schema documentation
- Canvas Designer Guide: Complete Canvas usage guide
- Java-Centered Types: Java-first approach with protobuf mapping
- Publishing to Maven Central: Guide to releasing and publishing the framework
This approach reduces boilerplate code and ensures consistency across your pipeline steps.