Cloudflare queues are really straight forward to setup.
- create the queue in the cf dashboard or via wrangler, and grab the queue name
- in any given cf worker project, you can define a queue
consumer
, which lets you listen to queue events, which are just arrays of arbitary json which you can define. pairing with zod we can manage complex message types. - in any given cf worker you can also define a
producer
which is the thing that can produce item to place into the queue. when setting up a producer you create a binding which can we called anything. in the Pinecone course we simply call itQUEUE
, which gives us an worker object which we can use anywhere in our cf worker to create a message like soc.env.QUEUE.send(queueMessage)
you can do anything when you receive the message such writing to d1 or invoking something else. queues open up the paradigm of background tasks.
CF also allows for more advanced queueing strategies such as dead letter queue
, this the ability to manage failed consumption of a message, e.g if the processing threw an error. it can automatially be placed into another queue.
You have lots of options of how to process message in the dead letter queue, it can be done within the same worker or completely different one.