I've been experiencing issues with LANGFLOW_WORKERS and cache configurations in Langflow

[muralimopidevi]

Bug Description

My current configuration:
langflow:
image: langflowai/langflow:latest
container_name: langflow
environment:
- LANGFLOW_WORKERS=4
- LANGFLOW_CONFIG_DIR=/app/langflow
- DO_NOT_TRACK=true
- BROKER_URL=amqp://langflow:langflow@rabbitmq:5672
- LANGFLOW_CACHE_TYPE=memory
- LANGFLOW_DATABASE_URL=postgresql://langflow:langflow@db_database:5432/langflow_db
- LANGFLOW_HOST=0.0.0.0
- LANGFLOW_PORT=7860
- LANGFLOW_LOG_FILE=logs/langflow.log
- LANGFLOW_OPEN_BROWSER=false
- LANGFLOW_REMOVE_API_KEYS=false
- LANGFLOW_AUTO_LOGIN=false
- LANGFLOW_SUPERUSER=user
- LANGFLOW_SUPERUSER_PASSWORD=Somepassword
- LANGFLOW_STORE_ENVIRONMENT_VARIABLES=true
- BACKEND_URL=http://localhost:7860/
volumes:
- langflow_data:/app/langflow

Problem Description
When LANGFLOW_WORKERS=1, everything works smoothly without any issues but it's very slow UI and response in playground or chat.
However, after increasing the worker count to LANGFLOW_WORKERS=4, the following issues arise:
Cache Configuration Problems:
When using LANGFLOW_CACHE_TYPE=memory, I encounter the error: "no cache found" in the playground while trying the flow.
When switching to LANGFLOW_CACHE_TYPE=redis, the error message shows: "Error building Component: Cannot pickle 'generator' object", along with the note "Redis Cache only accepts values that can be pickled".
The same issue occurs when using LANGFLOW_CACHE_TYPE=async.
It appears that when the number of workers increases, the cache configuration behaves inconsistently and results in errors.

Request for Assistance
I would greatly appreciate any guidance or clarification on how to resolve the issues with the cache configurations, especially when using Redis or asynchronous caching methods. It would also be helpful to know if there is an optimal way to handle scaling with more workers in Langflow.

Thank you in advance for your help!

Reproduction

services:

PostgreSQL Service

db_database:
image: postgres:16
container_name: db_database
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=password
- POSTGRES_DB=langflow_db
ports:
- "5432:5432"
command: >
postgres -c 'max_connections=${POSTGRES_MAX_CONNECTIONS:-100}'
-c 'shared_buffers=${POSTGRES_SHARED_BUFFERS:-128MB}'
-c 'work_mem=${POSTGRES_WORK_MEM:-4MB}'
-c 'maintenance_work_mem=${POSTGRES_MAINTENANCE_WORK_MEM:-64MB}'
-c 'effective_cache_size=${POSTGRES_EFFECTIVE_CACHE_SIZE:-4096MB}'
volumes:
- database:/var/lib/postgresql/data
networks:
- langflow_network

pgAdmin Service

pgadmin:
image: dpage/pgadmin4
container_name: pgadmin
environment:
- PGADMIN_DEFAULT_EMAIL=[email protected]
- PGADMIN_DEFAULT_PASSWORD=PassWord@123456789
ports:
- "5050:80"
volumes:
- pgadmin_data:/var/lib/pgadmin
networks:
- langflow_network

Langflow Service

langflow:
image: langflowai/langflow:latest
container_name: langflow
environment:
- LANGFLOW_WORKERS=4
- LANGFLOW_CONFIG_DIR=/app/langflow
- DO_NOT_TRACK=true
- BROKER_URL=amqp://langflow:langflow@rabbitmq:5672
- LANGFLOW_CACHE_TYPE=memory
- LANGFLOW_DATABASE_URL=postgresql://langflow:langlfow@db_database:5432/langflow_db
- LANGFLOW_HOST=0.0.0.0
- LANGFLOW_PORT=7860
- LANGFLOW_LOG_FILE=logs/langflow.log
- LANGFLOW_OPEN_BROWSER=false
- LANGFLOW_REMOVE_API_KEYS=false
- LANGFLOW_AUTO_LOGIN=false
- LANGFLOW_SUPERUSER=murali
- LANGFLOW_SUPERUSER_PASSWORD=XnKYvPnzPw?QFtbj
- LANGFLOW_SECRET_KEY=H37V\keDtc2bK'0tg4b>5GN7jo*3QZy1kTFIQ7$BNs49DWj!Dvq9XB~(L"\q<OfUoK23sz075$[8=5Bzad_s8h4zH5uH0V3aDJ
- LANGFLOW_STORE_ENVIRONMENT_VARIABLES=true
- BACKEND_URL=http://localhost:7860/
volumes:
- langflow_data:/app/langflow
depends_on:
- rabbitmq
- db_database
- langfuse
ports:
- "7860:7860"
networks:
- langflow_network

RabbitMQ Service

rabbitmq:
image: rabbitmq:3-management
container_name: rabbitmq
environment:
- RABBITMQ_DEFAULT_USER=langflow
- RABBITMQ_DEFAULT_PASS=langflow
ports:
- "5672:5672"
- "15672:15672"
volumes:
- rabbitmq_data:/var/lib/rabbitmq
networks:
- langflow_network

# Redis 

redis:
image: redis:6-alpine
restart: unless-stopped
container_name: redis
volumes:
- redis_data:/data
command: redis-server --requirepass redispassword
healthcheck:
test: ["CMD", "redis-cli", "ping"]
networks:
- langflow_network

Check Volumnes path

volumes:
database:
driver: local
driver_opts:
type: none
o: bind
device: /path/database
pgadmin_data:
driver: local
driver_opts:
type: none
o: bind
device: /path/pgadmin_data
langflow_data:
driver: local
driver_opts:
type: none
o: bind
device: /pathlangflow_data

networks:
langflow_network:
driver: bridge

Expected behavior

Langflow should be able to handle multiple workers efficiently to improve the speed and scalability of the backend processes.

Specifically:
When configuring LANGFLOW_WORKERS=4 (or more), the system should distribute the workload across multiple workers without encountering any errors.
Cache configurations such as memory, redis, and async should work seamlessly with multiple workers, without producing errors related to cache availability or object pickling.
Redis cache, in particular, should handle all types of serializable data, allowing Langflow to utilize caching mechanisms effectively to boost performance.

These are the bugs I have encountered so far, and I believe sharing them might be helpful for the community to resolve these issues and improve Langflow. Addressing these could make Langflow a more robust product:

Cache Issues with Multiple Workers:

When increasing LANGFLOW_WORKERS to 4, the flow fails with cache-related errors such as "no cache found".
Redis cache throws an error: "Error building Component: Cannot pickle 'generator' object", with the message "Redis Cache only accepts values that can be pickled".
Inconsistent Behavior with Cache Types:

Using different cache types like memory, redis, or async in combination with multiple workers causes system instability and prevents Langflow from functioning as expected.

Who can help?

No response

Operating System

Ubuntu Server 24.04 LTS

Langflow Version

1.0.18

Python Version

3.11

Screenshot

No response

Flow File

No response

[dosubot]

Hey @muralimopidevi, I'm here to help you with the issues you're experiencing with Langflow. Let's work together to resolve them!

To resolve the cache configuration issues in Langflow when increasing LANGFLOW_WORKERS to 4, especially with Redis and asynchronous caching methods, follow these steps:

1. Redis Cache Configuration:
   Ensure that the Redis cache is properly configured and connected. The relevant part of the code handles different cache types but raises a ConnectionError if the Redis cache is not connected. Here is the code snippet:

   if settings_service.settings.cache_type == "redis":
       logger.debug("Creating Redis cache")
       redis_cache: RedisCache = RedisCache(
           host=settings_service.settings.redis_host,
           port=settings_service.settings.redis_port,
           db=settings_service.settings.redis_db,
           url=settings_service.settings.redis_url,
           expiration_time=settings_service.settings.redis_cache_expire,
       )
       if redis_cache.is_connected():
           logger.debug("Redis cache is connected")
           return redis_cache
       else:
           raise ConnectionError("Failed to connect to Redis cache")

   Ensure that the Redis settings (redis_host, redis_port, redis_db, redis_url, redis_cache_expire) are correctly configured in your settings_service. If the Redis cache is not connected, the system will raise a ConnectionError ^[1][2].

2. RedisCache Class Implementation:
   The RedisCache class handles connection and serialization using the redis-py package and the pickle module. Ensure that the values you are trying to cache can be pickled. Here is the relevant part of the class:

   async def set(self, key, value, lock=None):
       try:
           if pickled := pickle.dumps(value):
               result = self._client.setex(str(key), self.expiration_time, pickled)
               if not result:
                   raise ValueError("RedisCache could not set the value.")
       except TypeError as exc:
           raise TypeError("RedisCache only accepts values that can be pickled.") from exc

   This method ensures that only values that can be pickled are set in the Redis cache ^[3][4].

3. ThreadingInMemoryCache Limitations:
   The ThreadingInMemoryCache class is suitable for single-threaded or multi-threaded applications within a single process but has limitations when used with multiple workers. Each worker will have its own instance of the cache, leading to inconsistencies. Additionally, the use of locks can lead to contention and reduced performance under high concurrency ^[4][3][5].

4. General Recommendations:

   • Verify that the Redis server is running and accessible from your application.
   • Check network connectivity and firewall settings to ensure that the Redis server can be reached.
   • Ensure that the Redis server configuration matches the settings provided in settings_service.

By ensuring these configurations and understanding the limitations of the in-memory cache, you should be able to resolve the cache configuration issues when increasing LANGFLOW_WORKERS to 4.

To continue talking to Dosu, mention @dosu.