Persistent Queuing - E-Commerce - Lecture Slides, Slides of Fundamentals of E-Commerce

E-Commerce is taking over the traditional commerce practices. It is of special concern for the IT students. Following are the key points of these Lecture Slides : Persistent Queuing, Explicit Traffic, Safe Place, Application, Transactional Resource, Sem Antics, Server Program, Client, Client Runs, Client Recovery

Typology: Slides

2012/2013

Uploaded on 07/30/2013

shoki_sho
shoki_sho 🇮🇳

4.9

(7)

121 documents

1 / 7

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
1
2/22/05 1
9. Queued Transaction
Pro cessing
545 Tran saction Pro ce ssing
2/22/05 2
Outline
1. Introduction
2. Transactional Sem antics
3. Q ueue Manager
A ppendices
A. Marsha ling
B. Multi-tran saction R equests (workflow )
C. (A ppendix) Microsoft M essag e Queue
2/22/05 3
9.1 Introduction
•Direct TP -a clien t sends a request to a server, waits
(synchronou sly) for the server to run the transaction
and possibly return a reply (e.g., RPC)
•Problem s with Direct TP
–Server or client-serve r com m unications is dow n
when the client wants to sen d the req u e st
–Client or client-serv er com m unications is dow n
when the serv er wants to send the rep ly
–If the serv er fa ils, how does the client find out what
happened to its outstanding requests?
Load balancing across many serv ers
–Priority-based scheduling of busy serv ers 2/22/05 4
Persisten t Queuing
Q ueuing -controlling work requests by moving
them through persistent transactional queues
B enefits of queuing
–clien t can send a re q u es t to an unavailable serv er
–server can send a rep ly to an unavailable client
–since the queue is persistent, a client can (in principle)
find out the state of a requ est
can dequeue req u e sts based on priority
can have many se rvers feed off a single queue
Client Server
Enqueue Dequeue
2/22/05 5
Other Benefits
Q ueue manager as a protocol gatew ay
need to support multiple protocols in just one system
env iro n m e n t
can be a trusted client of other system s to bridge secu rity
barriers
Explicit traffic control, without message loss
•Safe place to do m essa g e translation betw een
application formats
2/22/05 6
9.2 Transaction Sem antics Server View
•The queue is a transactional resou rce manager
•Server dequeues request within a tran saction
•If the transaction abo rts, the dequeue is undone,
so the request is returned to the queue
Start
Dequeue(Req, Q1)
process request Req
Enqueue(Reply, Q2)
Commit
Client
Enqueue
Dequeue
Server’s request queue
Client’s
reply queue
Server Program
Q1
Q2
Docsity.com
pf3
pf4
pf5

Partial preview of the text

Download Persistent Queuing - E-Commerce - Lecture Slides and more Slides Fundamentals of E-Commerce in PDF only on Docsity!

2/22/ 05 1

9.Q ueued Transaction

Processing

545 Transaction Processing

2/22/ 05 2

O utline

1.Introduction

2.TransactionalSem antics

3.Q ueue M anager

A ppendices

A .M arshaling

B.M ulti-transaction Requests (w orkflow )

C.(A ppendix)M icrosoftM essage Q ueue

2/22/ 05 3

9.1 Introduction

  • D irectTP -a clientsends a requestto a server,w aits

(synchronously)forthe serverto run the transaction

and possibly return a reply (e.g.,RPC)

  • Problem s w ith D irectTP
    • Serverorclient-servercom m unications is dow n w hen the clientw ants to send the request
    • Clientorclient-servercom m unications is dow n w hen the serverw ants to send the reply
    • Ifthe serverfails,how does the clientfind outw hat happened to its outstanding requests?
    • Load balancing across m any servers
    • Priority-based scheduling ofbusy servers (^) 2/22/ 05 4

PersistentQ ueuing

  • Q ueuing -controlling w ork requests by m oving

them through persistenttransactionalqueues

  • Benefits ofqueuing
    • clientcan send a requestto an unavailable server
    • servercan send a reply to an unavailable client
    • since the queue is persistent,a clientcan (in principle) find outthe state ofa request
    • can dequeue requests based on priority
    • can have m any servers feed offa single queue

Client (^) Server Enqueue (^) Dequeue

2/22/ 05 5

O therBenefits

  • Q ueue m anageras a protocolgatew ay
    • need to supportm ultiple protocols in justone system environm ent
    • can be a trusted clientofothersystem s to bridge security barriers
  • Explicittraffic control,w ithoutm essage loss
  • Safe place to do m essage translation betw een

application form ats

2/22/ 05 6

9.2 Transaction Sem antics ServerV iew

  • The queue is a transactionalresource m anager
  • Serverdequeues requestw ithin a transaction
  • Ifthe transaction aborts,the dequeue is undone,

so the requestis returned to the queue

Start Dequeue(Req, Q1) process request Req Enqueue(Reply, Q2) Commit

Client^ Enqueue Dequeue

Server’s request queue

Client’s reply queue

Q1 Server Program

Q

2

2/22/ 05 7

Transaction Sem antics

ServerV iew (cont’d)

  • Serverprogram is usually a w orkflow controller
  • Itfunctions as a dispatcherto
    • geta request,
    • callthe appropriate transaction server,and
    • return the reply to the client.
  • A bort-countlim itand errorqueue to dealw ith

requests thatrepeatedly lead to an aborted

transaction

2/22/ 05 8

  • Clientruns one transaction to enqueue a requestand

a second transaction to dequeue the reply

Transaction Sem antics-ClientV iew

Txn1: Start get input construct request Enqueue(Request, Q1) Commit Txn3: Start Dequeue(Reply, Q2) decode reply process output Commit

Q

Q

Txn2: Start Dequeue(Req, Q1) process request Req Enqueue(Reply, Q2) Commit

2/22/ 05 9

Transaction Sem antics

ClientV iew (cont’d)

  • Clienttransactions are very lightw eight
  • Still,every requestnow requires 3 transactions,

tw o on the clientand one on the server

  • M oreover,ifthe queue m anageris an independent resource m anager(ratherthan being partofthe database system ),then Transaction 2 requires tw o phase com m it
  • So queuing’sbenefits com e ata cost

2/22/ 05 10

ClientRecovery

  • Ifa clienttim es outw aiting fora reply,itcan

determ ine the state ofthe requestfrom the queues

  • requestis in Q 1,reply is in Q 2,orrequestis executing
  • A ssum e each requesthas a globally unique ID
  • Ifclientfails and then recovers,a requestcould be

in one of4 states:

  • A .Txn1 didn’tcom m it– no m essage in eitherqueue.
  • B.Txn1 com m itted butserver’s Txn2 did not– requestis eitherin requestqueue orbeing processed
  • C.Txn2 com m itted butTxn3 did not– reply is in the reply queue
  • D .Txn3 com m itted – no m essage in eitherqueue

2/22/ 05 11

ClientRecovery (2)

  • So,ifthe clientknow s the requestid R,itcan

determ ine state C and m aybe state B.

  • W hatifno queued m essage has the id R?

Could be in state A ,B,orD.

  • Can furtherclarify m atters ifthe clienthas a local

database thatcan run 2-phase com m itw ith the

queue m anager

  • U se the localdatabase to store the state ofthe request

2/22/ 05 12

Transaction Sem antics-ClientV iew

Txn1: Start Get Request R from local DB Enqueue(Request R, Q1) State(R) = “Submitted” Commit Txn3: Start Dequeue(Reply for R, Q2) decode reply & process output State(R) = “Done” Commit

Q

Q

Txn2: Start Dequeue(Req, Q1) process request Req Enqueue(Reply, Q2) Commit

Txn0: Start construct request & store it in local DB State(R) = “NotSubmitted” Commit

Notin the textbook

4

2/22/ 05 19

O ptim izations

  • In effect,the previous procedure m akes the action “process output” idem potent.
  • If“process output” senta m essage,itm ay notbe testable,so m ake sure it’s idem potent! - iftxn3 is sending a receipt,labelitby the serialnum berof the request,so itcan be senttw ice
  • Log device state as partofD equeue operation (saves an I/O ) - i.e.,run step 3 before step 2

2/22/ 05 20

9.3 Q ueue M anager

  • A queue supports m ostfile-oriented operations
    • create and destroy queue database
    • create and destroy queue
    • show and m odify queue’s attributes (e.g.security)
    • open-scan and get-next-elem ent
    • enqueue and dequeue
      • nextelem entorelem entidentified by index
      • inside oroutside a transaction
    • read elem ent

2/22/ 05 21

Q ueue M anager(cont’d)

  • A lso has som e com m unication types ofoperations
    • startand stop queue
    • volatile queues (lostin a system failure)
    • persistentsessions (explained earlier)
  • System m anagem entoperations
    • m onitorload
    • reporton failures and recoveries

2/22/ 05 22

Exam ple ofEnqueue Param eters

(IBM M Q Series)

  • System -generated and application-assigned m essage Ids
  • N am e ofdestination queue and reply queue (optional)
  • Flag indicating ifm essage is persistent
  • M essage type -datagram ,request,reply,report
  • M essage priority
  • Correlation id to link reply to request
  • Expiry tim e
  • A pplication-defined form attype and code page (forI18N )
  • Reportoptions -confirm on arrival(w hen enqueued)?, on delivery (w hen dequeued)?,on expiry?,on exception?

2/22/ 05 23

Priority O rdering

  • Prioritize queue elem ents
  • D equeue by priority
  • A bortm akes strictpriority-ordered dequeue too

expensive

  • could neverhave tw o elem ents ofdifferentpriorities dequeued and uncom m itted concurrently
  • Butsom e system s require itforlegalreasons
  • stock trades m ustbe processed in tim estam p order

2/22/ 05 24

Routing

  • Forw arding ofm essages betw een queues
    • transactional,to avoid lostm essages
    • batch forw arding ofm essages,forbetterthroughput
    • can be im plem ented as an ordinary transaction server
  • O ften,a lightw eightclientim plem entation supports

a clientqueue,

  • captures m essages w hen clientis disconnected,and
  • forw ards them w hen com m unication to queue serveris re-established
  • Im plies system m gm trequirem entto display

topology offorw arding links

5

2/22/ 05 25

State ofthe A rt

  • A llapp servers supportsom e form ofqueuing
  • A new trend is to add queuing to the SQ L D BM S
    • O racle has it.A voids 2PC forTxn2,allow s queries,….
  • Q ueuing is hard to build w ell.It’s a productor

m ajorsub-system ,notjusta feature.

  • Lots ofqueuing products w ith sm allm arketshare.
  • Som e m ajorones are
    • IBM ’s M Q Series
    • BEA System s M essageQ
    • M icrosoftM essage Q ueuing

2/22/ 05 26

A ppendix A :M arshaling

  • CallerofEnqueue and D equeueneeds to m arshaland unm arshaldata into variables
  • Instead,use the autom atic m arshaling ofRPC
  • H ere’s how RPC w orks:

A pp Proxy Runtim e (^) Runtim e Stub A pp callP

return to caller

pack argu- m ents

unpack results

send

receive

receive

send

unpack argu- m ents

pack results

P

w ork

return

w ait

Client’s System Server’sSystem Call packet

Return packet

2/22/ 05 27

A dapting RPC M arshaling forQ ueues

A pp Proxy Runtim e (^) Runtim e Stub A pp callP

return to app

pack argu- m ents

unpack results

enqueue request

dequeue reply

dequeue request

enqueue reply

unpack argu- m ents

pack results

P

w ork

return

w ait

Client’s System (^) requestserver Server’sSystem queue

client reply queue

  • In effect,use queuing as a transportforRPC
  • Exam ple – Q ueued Com ponentin M SM Q

2/22/ 05 28

A ppendix B:M ulti-Transaction Requests

  • Som e requests cannotexecute as one transaction because
    • Itexecutes too long (causing lock contention)or
    • Resourcesdon’tsupporta com patible 2-phase com m itprotocol.
  • Transaction m ay run too long because
    • Itrequiresdisplay I/O w ith user
    • People orm achines are unavailable (hotelreservation system , m anagerw ho approves the request)
    • Itrequires long-running real-w orld actions (get2 estim ates before settling an insurance claim )
  • Transaction m ay be required to run independentA CID transactions in subsystem s (placing an order,scheduling a shipm ent,reporting com m ission)

2/22/ 05 29

W orkflow

  • A m ulti-transaction requestis called a w orkflow
  • Integrated w orkflow products are now being offered.
    • IBM M Q SeriesW orkflow ,M S BizTalk O rchestration, TIBCO ,JetForm ,BEA W ebLogicProcess Integrator,A ction,...
    • See also w w w .w orkflow softw are.com ,w w w .w fm c.org
  • They have specialfeatures,such as
    • flow graph language fordescribing processes consisting ofsteps, w ith preconditions form oving betw een steps
    • representation oforganizationalstructure and roles (m anualstep can be perform ed by a person in a role,w ith com plex role resolution procedure)
    • tracing ofsteps,locating in-flightw orkflow s
    • ad hoc w orkflow ,integrated w ith e-m ail(case m gm t) 2/22/ 05 30

M anaging W orkflow w ith Q ueues

  • Each w orkflow step is a request
  • Send the requestto the queue ofthe serverthatcan process the request
  • Serveroutputs request(s)forthe nextstep(s)ofthe w orkflow

Submit expense claim

Validate claim

Get Manager Approval

Authorize Payment

Request Automatic Deposit

Email notification

7

2/22/ 05 37

M SM Q Interoperation

  • Exchange Connector-Send and receive m essages

and form s through Exchange Serverand M SM Q

  • M A PItransport-Send and receive m essages and

form s through M A PIand M SM Q

  • V ia Level8 System s,
    • Clients -M V S,A S/400,V M S,H P-U nix,Sun-Solaris, A IX ,O S/2 clients
    • Interoperates w ith IBM M Q Series