资源描述
Click to edit Master title style,Click to edit Master text styles,Second level,Third level,Fourth level,Fifth level,10/2/2010,#,Massaging the Pony:Message Queues and You,Shawn Rider,PBS Education,DjangoCon 2010,WhatisaMessageQueue?,Asystemforenablingasynchronousprocessingofdiscretetasks.,Superawesome.,Whataretheygoodfor?,Alleviateserverissuesbyoffloadingprocessing,Whataretheygoodfor?,Makebetteruserexperiences,Whataretheygoodfor?,Increasethereliabilityofthirdpartyserviceintegrations.,What are theygood for?,Background media processing,What are theygood for?,Repeating Tasks can replaceCron jobs,Keep all yourcode inyour project,Available MQ Solutions,There are manysolutions outthere.To name a fewof them:,RabbitMQ,AmazonSimpleQueue System,ApacheMQ,Gearman,OpenAMQ,Peafowl,Q4M,Starling,Sun Java System Message Queue,MessageQueueProtocols,AMQP: AdvancedMessage Queuing Protocol,JMS: Java Messaging Service,STOMP:Streaming TextOriented Messaging Protocol,Criteria for Broker Selection,The followingcriteria wereimportant to us in selectinga solution. You mayhave some different needs.,Handling exceptions and recovery,Low level of requiredmaintenance,Ease ofdeployment,Persistent queuing,Community support,What languageis it writtenin? Howcompatible isthat with ourcurrent systems?,Clustersupport,Failsafe MQ Brokers,Must beable to survive a server failure.,Upon restarting the MQ Broker, processingshouldresumefrom the lastun-processed messagesin queues,Queue Durability,The Messages Queue Broker must be able toreconstruct the queues whenit is restarted.,MessagePersistence,The Messages (tasks to be completed) mustpersistbetween shutdowns and restarts ofMQ Server,Our Choice,Celery(a Django taskmanagement app),Carrot(Djangomiddleware),PyAMQP(Pythonmodule),RabbitMQ (MQ Broker software),The ProductionSystem,Webapp,Database,Worker,Server(s),Celery,Rabbit,MQ,MQ in Development Environs,It is important to beable to develop Message Queue tasks in Django without being concernedaboutrunningthe broker process,queue setup, persistence etc.,MQ in Development Environs,Carrotcan beset upto usedifferent queuing backend for developmentpurposes,GhettoQprovides database backend for carrot where a database serves as the queue storage,The broker isa simple Django applicationthat monitorsthe queue in DB,Inefficient, therefore onlysuitable for development environments,CARROT_BACKEND= qhettoq.taproot.Database,Ease afutureMQ implementation,Isolatefunctionalityinto reusablecomponents.,RecycleExisting Code,Django,HTTP Requests& MQ,Most functionalitiesprovided throughDjangoapplicationsare tightly coupled with HTTP requests e.g. formdata processing,In order to asynchronously executea task,Celery(any task managementplatform) must serialize &store parameters,HTTP requestsare usually large andnot easily serializable. e.g. WSGIrequests,ThePickleableHTTPRequestObject,UsingPickleableHttpRequest,if,request.GET.get(download):,try,:,result= EnrollmentCSVExportTask.delay(,PickleableHttpRequest(,request,attributes=,user,admin_current_organization,),),MakingMessageQueueTasks,Itisfast andeasy to adda decoratortospecificfunctionstocreate MessageQueuetasks.,from,celery.decorators,importtask,task,defadd(x,y):,returnx +y,Explicitly InheritfromCelery,sTask Object,Then What?,Some MessageQueuetaskscan complete withnonotification required.,Forothertasks:,Notifybyemail,UseanAJAXlistenertonotifyusers,Some other messaging,InReview,ChooseyourMQsolutionwisely,Setuparobustsystem,Leverageyour existing code,Replace troublespotsastheycome up,Profit!,Message QueuesMake LifeBetter,
展开阅读全文