















































Studia grazie alle numerose risorse presenti su Docsity
Guadagna punti aiutando altri studenti oppure acquistali con un piano Premium
Prepara i tuoi esami
Studia grazie alle numerose risorse presenti su Docsity
Prepara i tuoi esami con i documenti condivisi da studenti come te su Docsity
Trova i documenti specifici per gli esami della tua università
Preparati con lezioni e prove svolte basate sui programmi universitari!
Rispondi a reali domande d’esame e scopri la tua preparazione
Riassumi i tuoi documenti, fagli domande, convertili in quiz e mappe concettuali
Studia con prove svolte, tesine e consigli utili
Togliti ogni dubbio leggendo le risposte alle domande fatte da altri studenti come te
Esplora i documenti più scaricati per gli argomenti di studio più popolari
Ottieni i punti per scaricare
Guadagna punti aiutando altri studenti oppure acquistali con un piano Premium
The document consists of class notes and personal revisions. This section of the document covers the following topics: Virtual reference feedback tuning (VRFT), reinforcement learnign.
Tipologia: Appunti
1 / 55
Questa pagina non è visibile nell’anteprima
Non perderti parti importanti!
















































VIRTUAL REFERENCE^ FEEDBACK^ TUNING (VRET)
~ (^) direct main feature Setup :^ output feedback^ control^ of^ an^ uncertain^ linear^ system (discrete^ time^ and^ deterministic)^ represented by a^ transfer^ function tunable control action · (^) uncertain linear system y(t) = P(z) u(t) · (^) for the moment we are (discrete time) observable discrete^ not (^) considering anydisturbance output time^ t. f. P(t) is (^) regarded as un certain (^) , can be (^) any rational +f. uncertainty we^ want^ to counteract (^) by means of (^) adaptation Control (^) problem : we want to (^) regulate y(t) (^) tracking rit) (^) via an (^) 1-degree of freedom control scheme deciding based (^) on the ult tracking error r(t)te(t)& ,^ (1z^ , G) u^ PC^ -y(t) Controller is (^) a linear (^) dyn. System controller chosen among classe of controllers (^) parametrized (^) by a certain (^) parameter vector (^) o
C = [C(z , 8)^ :^ ge-I admissible domain^ for^ o
C(z, 8) is (^) linearly parametrized in o^ if^ : ((z, 8)^ =^01. B i (z) +^02. B2(z) +...^ +^ Om^ · pm(z)
given pi(t)^ transfer^ functions BASES TRANSFER FUNCTION ((z, 8) =^ OT.^ B(z) = B(z)T.^0 O · ( PIE) = (BR) vector of bass^ oin linear e = (OTB(z) :^ de^
basis t.^ f. (^) represent (^) delays C(70) =^ a +^ 02zt... + 8mz^ MH (^) = Maz-^ = zm, + 827m (^) +... Om i = 1 zm poles all in (^) zeros FIR is^ dense^ within^ all linear controller^. (^) Every linear^ controller^ is well (^) approximated (^) by
long as (^) m is large (^) enough order · (^) cons : (^) m in application (^) typically hat^ to^ be^ too^ large "there are ather basis tot (^) (Laguerre polynomials) that (^) are still dense but (^) requires still (^) order 6 with (^) poles to
Bi(z) =^1 B2(t)^ = B3(z) = z - 1 Z
proportional discretetime^ integrator differentiation e = (c(z ,8) (^) : (^) GEIR3] = (^) PID controller VRFT (^) optimize the^ controller^ according control (^) design
JMRCO) cannot be even (^) computed -^ Gir (^) Is not (^) achievable when P(z) is un certain Idea : (^) an (^) experiment on the (^) plant can be (^) performed -^ >^ data^ driven Approximate the^ MR^ controller^ using a^ raw^ source^ of^ knowledge-data experiment on^ P(z) ult) & P(z)^ [(t)^ i^ injected,^ j is (1) (^) y(1) measured
ü(z) (^) y(z) i (^) i (^) approximate MR (^) control by (N) (^) y(N) using data (^) only DV =^ I/0 data (^) collected from our (^) experiment DN = (^) [ü(D (^) , (^) y()) , ...,IN) (^) y(N] VRFT -^ enforce MR^ goals by (^) relying on^ DN We could use and^ indirect (^) approach to^ solve the (^) problem : Di P = (^) P(z) model for Plz) P(z) can^ be^ any system ~ min IlPCO0)
obtaining a^ a^ good controller for^ P(z) Ilpo-MJMR(
(model (^) order selection-huge input on the mismatch (^) ( For this^ reason we^ will^ use a direct^ approach (much wiser (^) usage of^ data) VRFT : data preprocessing (set point) VIRTUAL REFERENCE = signal ebtained (^) by processing measured (^) output (t) with inverse of Model Reference /whole batch of data is available ( r(i) = M(z)"y(i) i^ = (^1) , ....^ N^ always well^ defined^ in^ an^ off-line^ setup (whole batch^ of^ data^ is available)
M(z) desired (^) closed-loop I (^). f (^).^ +^ often (^) strictly proper e (^). g. (^) M(z) =^11
p =^ real^ pole 1pk1^ as^ Stability shift (^) operator
= ~ (^) r(i) =^ z -py(i) = (py(i
1 - P^ ppy(i) no issue (^) , off-line set^ up i (^) y(i)(i) data all (^) available (^1) y(1) = (1) (^2) y(z) = (^) (2) uli) (^) can be (^) computed and this is the^ case 3 i whatever (^) M(z) is i N (^) y(N) VIRTUAL ERROR =^ difference between^ virtual^ reference and the measured (^) output e (i) =^ v(i) - y(i) i^ =^1 , ... N rii) eli (^) ult) (^) y(t)
(1) (^) y(1) & ü(z) (^) y(z) i (^) i (N) (^) y(N) M(z) OBS :(i) has^ the (^) property that^ if^ it^ is^ used as^ set-point the^ desired (^) response for the (^) class (^) loop coincides with the measured Output (^) gli) , vi)^ set-point^ , desired^ output :^ M(z)^ (i)^ =^ M(z)^ (M(z)"gli)] = Gli virtual (^) , not used in (^) practice key Fact^ : A (^) good controller (i (^). e. that achieves the MR (^) design (^) goals Plz)CIz,^ d)^ = M1z)) is such that when^ it^ is fed^ by the virtual^ error eli)^1 +^ P(z)((z,^ 8) it must return(i) as control action to^ be (^) injected into^ the (^) system V
" = (^) C(z, 8) " (^) -P(z)
In (^) general (^) any notion^ of^ distance^ can be^ used Fur(e)
for He It can be used (^) pre-filtering before minimizing (i) (^) = ((z) uu(i) = (^) ((z)u(i) & (i) (^) y(i) = ((z)y(i) pre-filter : [il any user^ chosen^ ↑L^ (i)^ = L(z)(i) e(i) (^) digital filter (^) et(i) = ((z)e(i) by linearity ,^ are^ obtained^ via URFT (^) signal construction over il and (^) il Jur(0) =li)-Cl VRFT cost function^ defined over (^) pre-filtered (^) signal &^ N
OVR = (in(i) -^ C(z^ , 8) er(i)) argmin (^) Ni ((z) = 1 - original algorithm L(z) =^ additional^ degree of freedom (^) that (^) we are
to (^) use to tune VRFT 2 JvR(d) =^ Jurlo) how (^) good is the^ =P)Mw
Right now^ ,^ at^ an^ intuitive^ level^ an^ approximation^ ok^ but^ not^ perfect Next (^) goal : understand the level of (^) approximation exatly and^ use^ (12)^ to^ reduce^ it as much^ as (^) possible -^ optimal selection of^ the^ prefilter - significant to^ improve VRFT^ to a (^) huge extent We need to Introduce (^) FREQUENCY INTERPRETATION OF PEM IDENTIFICATION OBS :^ VRFT^ and C1z (^) , 0)^ linearly parametrized Ind (17,0) =^ o^ + B(z) o B1z) : ( rector of (^) basis function I Jur(0) (^) -i) Cz,ili) B)
YL(i) BL(z)ec(i)
Jur(o) = ( üli)
Is (^) CIz (^) , OvR) (vRFT (^) controller) a (^) good MR controller? vr = argmin(i) C,^ il^ Fur(0) prefitered (^) prefiltered
argmin/1(PM)l JMR(0) same formulation as a PEM^ Identification problem what's the (^) relationship between Jur(0)^ and (^) JMR(0) and ovr^ and^ Our^? Moreover (^) ,^ tune^ (12)^ so^ that^ the^ match^ is as^ good as (^) possible We can answare to^ these (^) questions Via PEM IDENTIFICATION Frequency interpretation^ of^ PEM^ identification arginally developed^ in^ the^ context^ of^ identification and (^) we will (^) initially study it in this context llater we will use (^) acquired (^) knowledge in (^) VRFT) Data (^) generating system. Let's consider a (^) generic system observable &
any portion^ of^ reality^ that input (^) output connets^ variables^ of^ interest u(i) (^) y(1) u(N) (^) yin) N Experiment : Do(u(i) , y(1)^. (^) ..., u(N)^ , y(N)] collection of^110 pairs
We want to write (^) M10) in (^) prediction form (^) lisolating the (^) predictable part (^) up to +-1) : M(0) :^ y(t) =^ G(z, d)u(t - d) +^ H(z (^) , (^) b)z(t) y(t)
u(t - d) (^1) step predictor for (^) generic M(0) Asymtotic (^) theory of^ PEM^ identification^ : almost (^) surely
No J (^) (0) = 1((y(t)y(t( + - (^1). (^) 0))] stationarity implies that stationary
· (^) Almost surely =^ convergence^ arises^ Irrespective^ of^ the^10 data ~ asymptotically all^ experiments^ lead^ to the (^) same result · uniformly in^ SN(0)-510KEN)^ vo N relations of JN(8) (for different realizations)
small N
a (^) unique curve (^) 510) N (^) convergence of^ and^ the^ biggest error^ over ~ (^) different realization o goes to (^) zero
of (^) In(0) Vo ↓ unique rate^ of^ convergence
The convergence happen^ with (^) probability I ( = (^) for all (^) possible experiment) and (^) uniformly in o
property to^ have^ convergence -of minimizers Ev = argmin SN(0) vargmin()
O minimizer minimizer^ of^ asymptotic
cost function In (^) practice it^ is often the case that (^) Nis (^) large (^) enough so that mismatch between In(d)^ and "old and between^ or^ and 8 is (^) negligible /we work^ under this (^) condition) -study properties^ of^ Flo)^ and these will (^) apply approximately toÖN^ andV(0) (^) irrespective of the collected data with^ an error that (^) is (^) negligible The result of^ PEM Identification (^) is the same (^) irrespective of the (^) particular dataset D" as (^) long as the number of data (^) points is (^) big enough Hp :^ from^ now^ on^ we^ will^ consider^ N^ big enough /200^ -^300 points) y The mismatch (^) between M104) and Mon) (^) is (^) neghigible and (^) so if we characterize the (^) mismatch between (^) M(84) and S (^) , we are also (^) characterizing the mismatch (^) between MION) and S. Frequency interpretation^
5(0) (^) = Gew)G, ~ (^) w(W) I (^) #le5w (^) , O) (^) dwHesw, o) du -T fundamental tool to (^) evaluate Melw) =^ XVw^ white (^) nolse model mismatch^ frequency by (^) frequency
between (^) Ho(z) and H(z,8) in the construction (^) of the cost function (^) frequency (^) by frequency
0 - 5 wwN10, 02) 1 - az - 1u(t) +^ z(t) (^2).^ MARY^ Ary^ model^ class MARY (0) : (^) y(t) = ay(t - 1) + bu(t - 1) + (^) z(t) o = [) can (^) you (^) guess what's^ going on^ with^ PEM^ ident^ in the^ two^ cases^?
Y2z (^1) - z I 1 - 42z-
↑u (w) =^3 Melw) M°:^ y(t) =bu(t)^ & & G (^) (z (^) ,0) =b (t) ~^ wi^ (0^ ,^ 52)^ H(z^ ,^ 8)^ =^1
(0)-bd the minimum is for Alo-o^ for^ ofi]
Glz (^) , EN) (^) NoGlz, % (^) = G^ primary result from a central (^) perspective
asymptotically Sis^ perfectly^
MARY (^) : y(t) = ay(t - 1) + bu(t- 1) + (^) z(t) (^) zwN(0 , +2)o = [] y(t) = az^ - y(t) +^ bz (^) u(t) + (^) z(t) (1-az^ (y(t) =^ bz^ u(t)^ +^ z(t) y(t) = bz- 1 - az^ -^ u(t) + 1 - az
-youcannot achive (^) G =^ Gand H^ simultanosse a partial (^) consistency?^ Glow)^ G^
Tu (^) bejw 2 5(0) & = (^) 1- (^) Yeju 1-asu 1 Gesw^. du a A(0)^0 B(0) 1 = X -[] A() =^0 minimum for^ A^ 8 -^ [3] -B(82) =^1 J (^) (0) = Alo) + (^) B(0) no (^) minimum trade off *^ SIE^ (le(t)+^ vielt- 1) + Velt2)+.. (^) )
between and 02 ~ (^88) + G(EN) v GG() = no (^) partial (^) consistency HON) H( % ) #Ho no consistency
2B) (Case 2 :^70 ° - : Glz (^) , 8 % = (^) G(t) but Hz8) + Ho(t) vo)
GlzEN) (^) NoGlz , 0 % G(z, 8) = Gi(z)
1G -^ Glo)"u-to-y model mismatch (^) weighed by Hopta
allows (^) us to understand the (^) impact of the (^) -to-y model mismatch in^ 5(0) Pu(w) X2^ -high signal to^ noise^ ratio^ -^ contribution^ of^ the^ disturbance = (0) (^) = 16 · Glo incondw in (^) J(0) can^ be (^) neglected -T at 80 EN (^510) =G - G10 % Ho (^) prada -T (^) choice of o allows (^) us to (^) tune underparametrization, mismatch^ mismatch^ over^ frequency
Mu(w) (^) frequency (^) by frequency weight to^ the model mismatch (Hew, 84)/2^ Lindication^ when^ the^ mismatch^ is^ small^ or^ large ( large ~^ mismatch^ at^ that^ frequency has^ huge importance !
((z) Gli(i) y(i)
° (eswy IHesw,)(u(w)dw^
07 : (^) y(t) = -2u(t) + 1 - e(t) biz" + b2z^
ARX :^ y(t) =
1 + a (^) , z^ +^ azz
-^ 24(t)^ + 1 +^ aiz"^ +^ azz 2 e(t) G(0) H(0)
different (^) error models. In (^) any case (^) we know that none of the models are able^ to^ fully describe Go^ since it^ is a4th order^ +.^ f^. del ARX model How (^) to decide the model to (^) use : Now we'll (^) try to (^) see in which (^) freg. the error is localed Frequency interpretation^ : X 16-Glöi)) (^) weighted (^) by model mismatch IHIEN)/ 1 DE ~ weight 1
~ (^) G-GlNO) (^) is uniform (^) over all w freg. we (^) expect (^) I 1 big^ error^ here^ small ARX" (^) weight 1 =^1 + a, ARXy
103z" (^) (1 + (^0). 432" + (^0). 05622 + (^0). 00232-3) 1 - (^3). 17" +^3.^5802
but (^) it's due to (^) logaritmic DE scale (^) (the error is still constant) ARY SYSTEM linear scale ARx is bad at low freq.^ but^ much^ closer then of at^ high freq. ARY SYSTEM DE We can not^ say which^ model^ is best^ , it^ depend on the^ objective, and^ on the^ We of^ the^ system We want make ARX (^) good at^ low^ frequencies (^) by using prefiltering GARY-ARX, (^1) R = L(z) =^ H(z, VARY ,