Puppeteering an AI - Interactive Control of a Machine-Learning based Artificial Dancer

Abstract

This paper describes the authors' first experiments in creating an artificial dancer whose movements are generated through a combination of algorithmic and interactive techniques with machine learning. This approach is inspired by the time honoured practice of puppeteering. In puppeteering, an articulated but inanimate object seemingly comes to live through the combined effects of a human controlling select limbs of a puppet while the rest of the puppet's body moves according to gravity and mechanics. In the approach described here, the puppet is a machine-learning-based artificial character that has been trained on motion capture recordings of a human dancer. A single limb of this character is controlled either manually or algorithmically while the machine-learning system takes over the role of physics in controlling the remainder of the character's body. But rather than imitating physics, the machine-learning system generates body movements that are reminiscent of the particular style and technique of the dancer who was originally recorded for acquiring training data. More specifically, the machine-learning system operates by searching for body movements that are not only similar to the training material but that it also considers compatible with the externally controlled limb. As a result, the character playing the role of a puppet is no longer passively responding to the puppeteer but makes movement decisions on its own. This form of puppeteering establishes a form of dialogue between puppeteer and puppet in which both improvise together, and in which the puppet exhibits some of the creative idiosyncrasies of the original human dancer.

Full text

Puppeteering an AI - Interactive Control of a
Machine-Learning based Artificial Dancer
Daniel Bisig
&HQWHUIRU'DQFH5HVHDUFK&RYHQWU\8QLYHUVLW\&RYHQWU\8QLWHG.LQJGRPV
HPDLODG#FRYHQWU\DFXN
,QVWLWXWHIRU&RPSXWHU0XVLFDQG6RXQG7HFKQRORJ\=XULFK8QLYHUVLW\RIWKH$UWV=XULFK
6ZLW]HUODQG
HPDLOGDQLHOELVLJ#]KGNFK

Ephraim Wegner
2IIHQEXUJ8QLYHUVLW\2IIHQEXUJ*HUPDQ\
HPDLOHSKUDLPZHJQHU#KVRIIHQEXUJGH
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

$EVWUDFW
7KLV SDSHU GHVFULEHV WKH DXWKRUV ILUVW
H[SHULPHQWV LQ FUHDWLQJ DQ DUWLILFLDO
GDQFHUZKRVHPRYHPHQWVDUHJHQHUDWHG
WKURXJKDFRPELQDWLRQRIDOJRULWKPLFDQG
LQWHUDFWLYH WHFKQLTXHV ZLWK PDFKLQH
OHDUQLQJ7KLVDSSURDFKLVLQVSLUHGE\WKH
WLPH KRQRXUHG SUDFWLFH RI SXSSHWHHULQJ
,Q SXSSHWHHULQJ DQ DUWLFXODWHG EXW
LQDQLPDWHREMHFWVHHPLQJO\FRPHVWROLYH
WKURXJKWKHFRPELQHGHIIHFWVRIDKXPDQ
FRQWUROOLQJVHOHFWOLPEV RIDSXSSHWZKLOH
WKH UHVW RI WKH SXSSHWV ERG\ PRYHV
DFFRUGLQJ WR JUDYLW\ DQG PHFKDQLFV ,Q
WKHDSSURDFKGHVFULEHGKHUHWKHSXSSHW
LV D PDFKLQHOHDUQLQJEDVHG DUWLILFLDO
FKDUDFWHU WKDW KDV EHHQ WUDLQHG RQ
PRWLRQ FDSWXUH UHFRUGLQJV RI D KXPDQ
GDQFHU$VLQJOHOLPERIWKLV FKDUDFWHU LV
FRQWUROOHG HLWKHU PDQXDOO\ RU
DOJRULWKPLFDOO\ ZKLOH WKH PDFKLQH
OHDUQLQJ V\VWHP WDNHV RYHU WKH UROH RI
SK\VLFV LQ FRQWUROOLQJ WKH UHPDLQGHU RI
WKH FKDUDFWHUV ERG\ %XW UDWKHU WKDQ
LPLWDWLQJ SK\VLFV WKH PDFKLQHOHDUQLQJ
V\VWHP JHQHUDWHV ERG\ PRYHPHQWV WKDW
DUHUHPLQLVFHQWRIWKHSDUWLFXODUVW\OHDQG
WHFKQLTXH RI WKH GDQFHU ZKR ZDV
RULJLQDOO\ UHFRUGHG IRU DFTXLULQJ WUDLQLQJ
GDWD 0RUH VSHFLILFDOO\ WKH PDFKLQH
OHDUQLQJ V\VWHP RSHUDWHV E\ VHDUFKLQJ
IRU ERG\ PRYHPHQWV WKDW DUH QRW RQO\
VLPLODUWR WKHWUDLQLQJ PDWHULDOEXW WKDWLW
DOVR FRQVLGHUV FRPSDWLEOH ZLWK WKH
H[WHUQDOO\FRQWUROOHGOLPE$VDUHVXOWWKH
FKDUDFWHUSOD\LQJ WKH UROHRI D SXSSHWLV
QR ORQJHU SDVVLYHO\ UHVSRQGLQJ WR WKH
SXSSHWHHU EXW PDNHV PRYHPHQW
GHFLVLRQV RQ LWV RZQ 7KLV IRUP RI
SXSSHWHHULQJ HVWDEOLVKHV D IRUP RI
GLDORJXHEHWZHHQ SXSSHWHHU DQG SXSSHW
LQZKLFK ERWKLPSURYLVH WRJHWKHU DQGLQ
ZKLFK WKH SXSSHW H[KLELWV VRPH RI WKH
FUHDWLYH LGLRV\QFUDVLHV RI WKH RULJLQDO
KXPDQGDQFHU
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


1. Introduction
7KHWZRDXWKRUVKDYHEHHQFROODERUDWLQJ
IRU PDQ\ \HDUV LQ WKH SURGXFWLRQ RI
GDQFH SHUIRUPDQFHV FRQWULEXWLQJ IURP
WKHLUVLGHJHQHUDWLYHV\VWHPVIRUGDQFHUV
WRLQWHUDFWZLWKDQGWKHUHE\LQIOXHQFHOLYH
JHQHUDWHG V\QWKHWLF PXVLF DQG LPDJHU\
2QH RI WKH PDLQ IDVFLQDWLRQV RI WKHVH
DFWLYLWLHV OLHV LQ WKH GHYHORSPHQW RI
JHQHUDWLYHSURFHVVHVWKDWDUHVXIILFLHQWO\
FRPSOH[ LQ WKHLU EHKDYLRXU WR DFKLHYH D
ZLGHUDQJHRIDHVWKHWLFUHVXOWVEXWDWWKH
VDPHWLPHFDQEHH[SRVHGWRDGDQFHULQ
D PDQQHU WKDW PDNHV WKHP UHDGLO\
XQGHUVWDQGDEOH DQG HQJDJLQJ WR
LPSURYLVHZLWK7KLVVDPHIRFXVDOVROLHV
DWWKHIRUHIURQWRIWKHUHVHDUFKSUHVHQWHG
KHUH EXW LQVWHDG RI DSSO\LQJ LW WR UXOH
EDVHG JHQHUDWLYH V\VWHPV LW QRZ
FRQFHUQVDPDFKLQHOHDUQLQJV\VWHPWKDW
KDGSUHYLRXVO\EHHQGHYHORSHGE\RQHRI
WKH DXWKRUV 7KLV V\VWHP ZLWK WKH QDPH
*UDQXODU 'DQFH >@ KDV RULJLQDOO\ EHHQ
GHYLVHG DV D FRFUHDWLYH WRRO IRU
FKRUHRJUDSKHUV WR JHQHUDWH V\QWKHWLF
PRWLRQ PDWHULDO WKDW LV UHPLQLVFHQW LQ
IRUP DQG VW\OH RI WKH PRYHPHQWV RI D
GDQFHUWKDWWKH V\VWHPKDGEHHQWUDLQHG
RQ :LWK WKLV SXUSRVH LQ PLQG *UDQXODU
'DQFH ZDV JHDUHG WRZDUGV D FUHDWLYH
ZRUNIORZ LQ ZKLFK WKH SRWHQWLDO IRU
H[SORUDWLRQ LGHDWLRQ DQG GLVFRYHU\ ZDV
DELJJHUFRQFHUQWKDQLQWXLWLYHLQWHUDFWLRQ
DQGUHDOWLPHIHHGEDFN
7KH FXUUHQW UHVHDUFK VWDUWHG DIWHU
*UDQXODU 'DQFH KDG EHHQ UH
LPSOHPHQWHGIRUUHDOWLPHDQGLQWHUDFWLYH
DSSOLFDWLRQ 7KLV UHLPSOHPHQWDWLRQ RIIHUV
DPRQJRWKHUV WKHSRVVLELOLW\ WR JHQHUDWH
PRWLRQVIRUDQDUWLILFLDOGDQFHUZLWKZKRP
D KXPDQ GDQFHU LQWHUDFWV ZLWK LQ DQ
LPSURYLVHG GXHW :KLOH WKH LPSURYHG
FRPSXWDWLRQDO SHUIRUPDQFH RI WKH
UHLPSOHPHQWHG V\VWHP PDNHV WKLV
VFHQDULR WKHRUHWLFDOO\ IHDVLEOH LWV
SUDFWLFDELOLW\ LV IDU IURP REYLRXV 7KLV LV
GXH WR SDUWLFXODULWLHV RI WKH PDFKLQH
OHDUQLQJ DUFKLWHFWXUH WKDW IRUPV SDUW RI
*UDQXODU'DQFHDQGWKHPHWKRGIRUPHUO\
HPSOR\HG IRU VHDUFKLQJ IRU DQG
JHQHUDWLQJ V\QWKHWLF PRWLRQV *UDQXODU
'DQFH HPSOR\V DPRQJ RWKHUV DQ
DXWRHQFRGHU WKDW RSHUDWHV RQ GDQFH
UHFRUGLQJVLQ WKH IRUPRIPRWLRQ FDSWXUH
GDWD 7KLV GDWD LV WKHQ HQFRGHG E\ WKH
DXWRHQFRGHU LQWR D FRPSUHVVHG
UHSUHVHQWDWLRQ LQ WKH IRUP RI ODWHQW
YHFWRUVZKLFK FDQEH GHFRGHGWR REWDLQ
D PRUH RU OHVV IDLWKIXO UHFRQVWUXFWLRQ RI
WKH RULJLQDO PRWLRQ GDWD 0RVW RI WKH
FUHDWLYH DSSOLFDWLRQV RI DXWRHQFRGHUV
LQFOXGLQJ WKH RQH RULJLQDOO\ FKRVHQ IRU
*UDQXODU 'DQFH GLVSRVH ZLWK WKH
HQFRGLQJ SURFHVV DQG LQVWHDG GLUHFWO\
VHDUFK IRU ODWHQW YHFWRUV WKDW PLJKW
SURGXFH LQWHUHVWLQJ PRWLRQV RQFH
GHFRGHG 7KLV VHDUFK IRU ODWHQW YHFWRUV
FRUUHVSRQGV WR D QDYLJDWLRQ LQ ODWHQW
VSDFH8QIRUWXQDWHO\LWLV GLIILFXOW WRJDLQ
DQ LQWXLWLYH XQGHUVWDQGLQJ IRU WKH
RUJDQLVDWLRQ RI WKLV ODWHQW VSDFH DQG WR
EHFRPH DEOH WR QDYLJDWH LW LQ D JXLGHG
PDQQHU 7KLV GLIILFXOW\ LV OLQNHG WR WKH
LQWUDFWDEOH UHODWLRQVKLS EHWZHHQ PRWLRQ
GDWD DQG LWV HQFRGLQJ 0RUHRYHU WKH
VSDWLDORUJDQLVDWLRQRIWKHODWHQWVSDFHLV
UDUHO\ UHODWHG WR SHUFHSWXDO DVSHFWV RI
WKH GDWD 7KLV PLJKW QRW EH PXFK RI DQ
LVVXH ZKHQ WKHUH LV DPSOH WLPH IRU
H[SORUDWLRQDQGH[SHULPHQWDWLRQDWRQH¶V
GLVSRVDO%XWIRUD SHUIRUPHU ZKRQHHGV
WR TXLFNO\ GHFLGH KRZ WR LQWHUDFW DQG
ZKDW WR H[SHFW IURP WKLV LQWHUDFWLRQ
QDYLJDWLQJ ODWHQW VSDFH LV DQ
RYHUZKHOPLQJO\GDXQWLQJWDVN
)RUWKLVUHDVRQWKHDXWKRUVKDYHVWDUWHG
WRH[SORUHRWKHUPHDQVRILQWHUDFWLQJZLWK
DQ DXWRHQFRGHU SDUWLFXODUO\ IRFXVLQJ RQ
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


IRUPV RI LQWHUDFWLYH FRQWURO WKDW WDNHV
SODFH RQ WKH OHYHO RI PRWLRQ LWVHOI UDWKHU
WKDQ LWV HQFRGLQJ 7KHVH H[SORUDWLRQV
KDYH OHG WR WKH GLVFRYHU\ WKDW
DXWRHQFRGHUV ZKHQ DSSOLHG LWHUDWLYHO\
RQ WKHLU RZQ GHFRGHG RXWSXW FDQ
SURGXFHLQWHUHVWLQJ V\QWKHWLFPRWLRQV LQ
SDUWLFXODU ZKHQ VRPH RI WKH YDOXHV RI
WKLV RXWSXW DUH UHPRYHG IURP WKH
DXWRHQFRGHU¶VLQIOXHQFHDQGVHWE\RWKHU
DOJRULWKPVRUWKURXJKLQWHUDFWLRQLQVWHDG
7KH IRFXV RI WKLV SDSHU OLHV RQ WKH
VHFRQGPHWKRGWKHLQWHUDFWLYHFRQWURORI
HOHPHQWV LQ D PRWLRQ FDSWXUH VHTXHQFH
ZKLFK LV DW WKH VDPH WLPH UHSHDWHGO\
HQFRGHG DQG GHFRGHG E\ WKH
DXWRHQFRGHU 6LQFH WKH DXWRHQFRGHU
SURGXFHV DQ RXWSXW WKDW VSHFLILHV WKH
PRWLRQVIRUWKHHQWLUHERG\RIDQDUWLILFLDO
GDQFHU LQWHUDFWLYHO\ FRQWUROOLQJ RQO\
VRPHMRLQWVRIWKLVERG\LVUHPLQLVFHQWRI
SXSSHWHHULQJLQ ZKLFKD SXSSHWHHU RQO\
FRQWUROV D VXEVHW RI D SXSSHW¶V MRLQWV
ZKLOHWKHRWKHUMRLQWVDUHWDNHQFDUHRIE\
DQRWKHU SURFHVV ,Q WUDGLWLRQDO
SXSSHWHHULQJ WKLV RWKHU SURFHVV LV
SK\VLFV ZKHUHDV LQ WKH V\VWHP
SUHVHQWHGKHUHLWVLVDXWRHQFRGLQJ:KLOH
SK\VLFVHQVXUHVWKDWWKHSXSSHWSHUIRUPV
SK\VLFDOO\ SODXVLEOH PRWLRQV
DXWRHQFRGLQJ HQVXUHV WKDW WKH DUWLILFLDO
GDQFHU H[KLELWV PRWLRQV WKDW DUH
VW\OLVWLFDOO\ SODXVLEOH WR WKH H[WHQW WKDW
WKH\ UHVHPEOH PRYHPHQWV WKDW KDYH
EHHQUHFRUGHGIURPDGDQFHU
7KHUHPDLQGHURIWKHSDSHULVRUJDQLVHG
DVIROORZV)LUVW LV DEDFNJURXQGVHFWLRQ
WKDW FRYHUV VRPH RI WKH PHWKRGV DQG
ILHOGV WKDW DUH UHODWHG WR WKH DXWKRUV¶
FXUUHQWUHVHDUFK7KLVLQFOXGHVWKHXVHRI
PRWLRQFDSWXUHWRGLJLWLVHWKHPRYHPHQWV
RI GDQFHUV WHFKQLTXHV IRU FUHDWLQJ
V\QWKHWLFPRWLRQVZLWKDVSHFLILFIRFXVRQ
FUHDWLYH DSSOLFDWLRQV LQ GDQFH DQG WKH
ILHOG RI GLJLWDO SXSSHWU\ WKDW UHOLHV RQ
LQWHUDFWLYH IRUPV RI PRWLRQ V\QWKHVLV
6XEVHTXHQWO\ D EULHI VXPPDU\ RI WKH
RULJLQDO *UDQXODU 'DQFHV\VWHPLV
SURYLGHG IROORZHG E\ D PRUH H[WHQVLYH
LQWURGXFWLRQRIWKHFKDQJHVLPSOHPHQWHG
DVSDUWRIWKHFXUUHQWUHVHDUFK7KHQWKH
H[SHULPHQWV FRQGXFWHG VR IDU ZLWK WKH
PRGLILHG *UDQXODU 'DQFH V\VWHP DUH
SUHVHQWHG DQG GLVFXVVHG )LQDOO\ WKHVH
UHVXOWVDUHSODFHGLQDZLGHUFRQWH[WZLWK
DQRXWORRNRQIXWXUHUHVHDUFKGLUHFWLRQV
2. Background
2.1 Motion Capture
0RWLRQ FDSWXUH LV D GLJLWDO WHFKQLTXH
ZKHUHE\ WKH PRWLRQV RI RQH RU VHYHUDO
SHUIRUPHUV DUH FDSWXUHG E\ PHDQV RI
UHIOHFWLYH RU PDJQHWLF PDUNHUV DWWDFKHG
WR WKHLU ERG\ SDUWV >@ $ YDULHW\ RI
VHQVLQJWHFKQRORJLHVFDQEHXVHGIRUWKLV
SXUSRVH UDQJLQJ IURP KLJK HQG RSWLFDO
V\VWHPV FRQVLVWLQJ RI PXOWLSOH FDPHUDV
VXUURXQGLQJ WKH SHUIRUPHUV WR PRUH
DIIRUGDEOHEXW DOVR OHVVSUHFLVH V\VWHPV
VXFK DV J\URVFRSHV DFFHOHURPHWHUV RU
ORZ HQG GLVWDQFH VHQVLQJ FDPHUDV 7KH
UHFRUGLQJV SURGXFH GLJLWDO GDWD LQ WKH
IRUP RI WLPHVHULHV RI VXUIDFH SRLQW
SRVLWLRQVZKLFKDUHWKHQW\SLFDOO\XVHGWR
UHFRQVWUXFWDQDEVWUDFWWKUHHGLPHQVLRQDO
UHSUHVHQWDWLRQ RI WKH SHUIRUPHUV¶ ERG\
SRVWXUHV 7KLV VR FDOOHG VNHOHWRQ
UHSUHVHQWDWLRQ FDQ WKHQ EH IXUWKHU
SURFHVVHG DQG YLVXDOLVHG WKURXJK D
YDULHW\ RI PHDQV ,Q WKH ILHOG RI GDQFH
PRWLRQFDSWXUHKDVEHFRPHDQLPSRUWDQW
WRRO IRU DFDGHPLF DQG DUWLVWLF SXUSRVHV
> @ $FDGHPLF DSSOLFDWLRQV RI PRWLRQ
FDSWXUH UHFRUGLQJV LQFOXGH WKHLU XVH DV
FRPSXWDWLRQIULHQGO\ FRPSOHPHQW WR
GDQFH QRWDWLRQ >@ DV UHVRXUFH IRU
PRWLRQ DQDO\VLV ZLWK WKH JRDO RI
H[WUDFWLQJ KLJK OHYHO TXDOLWDWLYH
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


LQIRUPDWLRQ IURP ORZ OHYHO SK\VLFDO
GHVFULSWRUV>@RUWKHGRFXPHQWDWLRQDQG
SUHVHUYDWLRQRI GDQFHWUDGLWLRQV VXFKDV
&\SULRW IRON GDQFH >@ &UHDWLYH
DSSOLFDWLRQVRIPRWLRQFDSWXUHLQFOXGHLWV
XVH IRU JHVWXUHEDVHG LQWHUDFWLRQ ZLWK
OLYH PHGLD GXULQJ SHUIRUPDQFH HJ
$SSDULWLRQ 6WRFRV DQG '|NN IRU
DQLPDWLQJ DUWLILFLDO GDQFHUV LQ YLUWXDO
UHDOLW\ HJ 'XVW 'D]]OH WHOHPDWLF
SHUIRUPDQFH HJ 7HOHPDWLF 7RXFK DQG
*R /D &RPHGLH 9LUWXHOOH FRPSXWHU
DQLPDWLRQ HJ $VSK\[LD 'LJLWDO %RG\
3URMHFW RU IRU FUHDWLQJ LQWHUDFWLYH
V\QWKHWLF GDQFHUV RQ VWDJH HJ
(PHUJHQFH$,DPKHUH
2.2 Motion Synthesis
7KH WHUP PRWLRQ V\QWKHVLV LV HPSOR\HG
KHUH IRU DQ\ DQLPDWLRQ WHFKQLTXH IRU
FRPSXWHU JHQHUDWHG FKDUDFWHUV WKDW
$SSDUDWLRQKWWSVZZZHVFDSHLQWROLIHFRPDUW
YLGHRVNODXVREHUPDLHUDSSDULWLRQ
6WRFRVKWWSVZZZVWRFRVFRPHQSDJHVWRFRV
'|NNKWWSVZZZIXVHZRUNVLWHQZRUNVGRNN
'XVW
KWWSVGLJLWDODUWDUFKLYHVLJJUDSKRUJDUWZRUNPDULD
MXGRYDDQGUHMEROHVODYVN\GXVW
'D]]OHKWWSVVSULQJEDFNPDJD]LQHFRPUHDGEIL
OIIH[SDQGHGGDQFHYLUWXDOUHDOLW\GRFXPHQWDU\
7HOHPDWLF7RXFKDQG*R
KWWSVMRXUQDOVJROGDFXNLQGH[SKSOHDDUWLFOHYLHZ

/D&RPHGLH9LUWXHOOH
KWWSVZZZJLOOHVMRELQFRPHQFUHDWLRQYLUWXDO
FRPHGLH
$VSK\[LD
KWWSVZZZWKLVLVFRORVVDOFRPDVSK\[LDD
VWULNLQJIXVLRQRIGDQFHDQGPRWLRQFDSWXUH
WHFKQRORJ
'LJLWDO%RG\3URMHFW
KWWSVZZZDOH[DQGHUZKLWOH\FRPGLJLWDOERG\
(PHUJHQFH
KWWSVMRKQPFFRUPLFNLQIRFDWHJRU\HPHUJHQFH
$,DPKHUH
KWWSVZZZ[RU[RUKXSURMHFWVWKHDWUHDLDPKWPO
GRHVQ¶W VROHO\ UHO\ RQ WKH SOD\EDFN RI
SUHYLRXVO\ UHFRUGHG PRWLRQ 7KH PRVW
SURPLQHQW DSSOLFDWLRQ GRPDLQ RI PRWLRQ
V\QWKHVLV LV FRPSXWHU DQLPDWLRQ DQG
JDPHGHVLJQ2WKHUILHOGVWKDWDOVRPDNH
XVH RI V\QWKHWLFDOO\ JHQHUDWHG PRWLRQV
DUH URERWLFV LQWHUDFWLRQ GHVLJQ DQG
GDQFH
,Q GDQFH WKH JHQHUDWLRQ RI V\QWKHWLF
PRWLRQ VHUYHV D YDULHW\ RI SXUSRVHV DV
VRXUFHIRULQVSLUDWLRQDQGLGHDWLRQGXULQJ
WKH FUHDWLYH SURFHVV DV PHFKDQLVP IRU
FRQWUROOLQJ DUWLILFLDO GDQFHUV WKDW DFW DV
LPSURYLVDWLRQ SDUWQHUV GXULQJ UHKHDUVDO
RUSHUIRUPDQFHRUWRKLJKOLJKWDQGVWXG\
FKRUHRJUDSKLF SULQFLSOHV WKDW PLJKW EH
GLIILFXOW WR FRPSUHKHQG ZKHQ UHO\LQJ RQ
YLGHRUHFRUGLQJVRQO\
6\QWKHWLF PRWLRQV FDQ EH FUHDWHG E\ D
YDULHW\ RI PHDQV LQFOXGLQJ SK\VLFV
VLPXODWLRQ DUWLILFLDO HYROXWLRQ DQG
PDFKLQHOHDUQLQJ
2.2.1 Physical Simulation
6LPXODWLRQV WKDW PRGHO ULJLG ERG\
G\QDPLFV RU PDVVVSULQJ V\VWHPV DUH
DWWUDFWLYH VLQFH WKH\ JHQHUDWH V\QWKHWLF
PRWLRQV WKDW DSSHDU SK\VLFDOO\ YDOLG
0DQ\ H[DPSOHV WKDW HPSOR\ SK\VLFV
VLPXODWLRQ IRU FUHDWLYH SXUSRVHV XVH
PDVVVSULQJ V\VWHPV WR FUHDWH DEVWUDFW
DUWLILFLDO ERGLHV WKDW SRVVHVV QRQ
DQWKURSRPRUSKLF PRUSKRORJLHV $
FKRUHRJUDSKLF VXSSRUW WRRO WKDW LV
LQVSLUHG E\ FKRUHRJUDSKLF WKLQNLQJ LQ
SK\VLFDO PHWDSKRUV FUHDWHV V\QWKHWLF
PRWLRQV IRU DEVWUDFW ERGLHV WKDW FRQVLVW
RI D PLQLPDO VHW RI PDVVSRLQWV DQG
VSULQJV >@ $Q LQWHUDFWLYH LQVWDOODWLRQ
HQWLWOHG %HFRPLQJHPSOR\VDPDVV
VSULQJ VLPXODWLRQ WR FUHDWH DQ DUWLILFLDO
ERG\ WKDW H[KLELWV VHOI PRWLYDWHG
PRYHPHQWV >@ 7KH DUWLILFLDO ERG\ LV
GLVSOD\HG DV DEVWUDFW JUDSKLFDO
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


DQLPDWLRQV PHDQW WR HYRNH NLQDHVWKHWLF
HPSDWK\ DQG WKHUHE\ IRVWHU PRYHPHQW
LGHDWLRQDPRQJGDQFHUVZKRUHKHDUVHLQ
SUHVHQFHRIWKHLQVWDOODWLRQ$VLPXODWLRQ
EDVHG V\VWHP WKDW KDV EHHQ HPSOR\HG
IRUHGXFDWLRQDOSXUSRVHVDQGLQDGDQFH
SLHFH FUHDWHV DQ LQWHUDFWLYH DUWLILFLDO
GDQFHU ZKRVH QRQDQWKURSRPRUSKLF
PRUSKRORJ\ LV PRGHOOHG DV PDVVVSULQJ
V\VWHP >@ 7KH UHVSRQVLYH EHKDYLRXUV
RIWKHDUWLILFLDOGDQFHUFRQYH\H[SUHVVLYH
PRYHPHQW TXDOLWLHV ZKLFK DUH REWDLQHG
E\ GHVLJQLQJ VSHFLILF PDSSLQJV WR
VLPXODWLRQ SDUDPHWHUV IRU HDFK RI WKHP
,QWKH 1HXUDO 1DUUDWLYHV VHULHVRI GDQFH
SLHFHV D VLPXODWHG PDVVVSULQJ V\VWHP
LVFRPELQHGZLWKDVLPSOHDUWLILFLDOQHXUDO
QHWZRUN WR FUHDWH V\QWKHWLF ERG\ OLPEV
WKDW DFWV DV DUWLILFLDO H[WHQVLRQV WR D
KXPDQGDQFHU¶VQDWXUDOERG\>@
2.2.2 Artificial Evolution
$UWLILFLDOHYROXWLRQKDVEHHQHPSOR\HGIRU
WKHFUHDWLRQRIV\QWKHWLFPRWLRQVWKDWFDQ
EH XVHG E\ FKRUHRJUDSKHUV DV
LQVSLUDWLRQDO UHVRXUFH $ V\VWHP HQWLWOHG
6FXGGOH FUHDWHV LQFRPSOHWH PRWLRQ GDWD
WKDW VHUYHV DV FDWDO\VWV IRU D
FKRUHRJUDSKHU¶VFUHDWLYLW\ >@ /DWHURQ
WKH VDPH UHVHDUFK WHDP KDV GHYHORSHG
D V\VWHP HQWLWOHG &RFKRUHR WKDW
JHQHUDWHV XQLTXH NH\IUDPHV DV VHHG
PDWHULDO IRU D FKRUHRJUDSKHU¶V FUHDWLYH
SURFHVV >@ %RWK RI WKHVH V\VWHPV
JHQHUDWH V\QWKHWLF PRWLRQ XVLQJ D
*HQHWLF$OJRULWKP ZKRVHILWQHVVIXQFWLRQ
LQFRUSRUDWHVDPRQJRWKHUV/DEDQ0RWLRQ
$QDO\VLVFDWHJRULHV>@7KH'DQFLQJ
*HQRPH 3URMHFW HPSOR\V DQ LQWHUDFWLYH
JHQHWLFDOJRULWKPWRPRGLI\VHTXHQFHVRI
EDVLF PRWLRQV ZKLFK DUH WKHQ VKRZQ DV
OLYH VFRUHV WR ERWK DQ DUWLILFLDO DQG D
KXPDQGDQFHU>@
2.2.3 Machine Learning
:LWK UHFHQW SURJUHVV LQ PDFKLQH
OHDUQLQJWKHXVHRIGDWDGULYHQPHWKRGV
IRU V\QWKHVLVLQJ PRWLRQ KDV JDLQHG LQ
SRSXODULW\ 'DWD GULYHQ PHWKRGV KDYH
SURYHQ HIIHFWLYH LQ JHQHUDWLQJ V\QWKHWLF
PRWLRQV WKDW DUH QDWXUDO ORRNLQJ DQG
H[SUHVVLYH 7KHVH PHWKRGV FDQ FDSWXUH
DQG LPLWDWH WKH LGLRV\QFUDWLF PRYHPHQW
VW\OHVRILQGLYLGXDO GDQFHUV ZKHQWUDLQHG
RQ FRUUHVSRQGLQJ PRWLRQ FDSWXUH
UHFRUGLQJV
6HYHUDOPDFKLQHOHDUQLQJV\VWHPVEDVHG
RQQHXUDOQHWZRUNVKDYHEHHQSURSRVHG
DV FRFUHDWLYH WRROV IRU FKRUHRJUDSKHUV
$V\VWHPHQWLWOHG&KRUUQQLPSOHPHQWVD
UHFXUUHQW QHXUDO QHWZRUN >@ 7KH
DXWKRUV RI WKLV V\VWHP VXJJHVW D
FROODERUDWLYH ZRUNIORZ LQ ZKLFK
FKRUHRJUDSKHU DQG WRRO WDNH WXUQV LQ
FUHDWLQJ PRWLRQ PDWHULDO ,Q WZR
SXEOLFDWLRQV GLIIHUHQW GHHSOHDUQLQJ
DUFKLWHFWXUHV DUH FRPSDUHG ZLWK UHJDUG
WR WKHLU XVHIXOQHVV IRU FKRUHRJUDSKLF
SXUSRVHV %DVHG RQ D VXEMHFWLYH
HYDOXDWLRQ RI PL[HG GHQVLW\ QHWZRUNV
DXWRHQFRGHUV DQG /RQJ6KRUW 7HUP
0HPRU\ /670 QHWZRUNV WKH DXWKRUV
FRQFOXGH WKDW /670V SHUIRUP EHVW RQ
FULWHULD VXFK DV SRVWXUH SUHGLFWLRQ
WHPSRUDOFRKHUHQFHPRWLRQFRQVLVWHQF\
DQGDHVWKHWLFV >@ $QRWKHUFRPSDULVRQ
EHWZHHQ DXWRHQFRGHUV DQG /670V
SODFHV D VWURQJHU IRFXV RQ WKH IOH[LELOLW\
RI FUHDWLQJ PRWLRQ YDULDWLRQV >@ 7KLV
FRPSDULVRQHQGVXSJLYHQPRUHDWWHQWLRQ
WRDXWRHQFRGHUVWKDQ/670V
0DFKLQHOHDUQLQJ V\VWHPV KDYH DOVR
EHHQ HPSOR\HG WR UHDOLVH LQWHUDFWLYH
DUWLILFLDOGDQFHUV6XFK DQ DSSURDFKKDV
EHHQWKRURXJKO\H[SORUHGE\%HUPDQDQG
-DPHV >@ DQG 0F&RUPLFN DQG
FROOHDJXHV >@ %RWK WHDPV
H[SHULPHQWHG ZLWK D YDULHW\ RI PDFKLQH
OHDUQLQJ WHFKQLTXHV WR REWDLQ D V\VWHP
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


FDSDEOHRIV\QWKHVLVLQJPRWLRQV WKDW DUH
VLPLODUWRWKRVHRIDKXPDQGDQFHU7KHLU
V\VWHPV KDG LQLWLDOO\ EHHQ HPSOR\HG LQ
LQWHUDFWLYH UHKHDUVDO VHWWLQJV DQG ZKHUH
ODWHURQ DGDSWHG IRU VWDJHSHUIRUPDQFHV
LQ ZKLFK DQ DUWLILFLDO DQG KXPDQ GDQFHU
LQWHUDFW LQ D GXHW 7KH /XPLQ$,V\VWHP
HPSOR\V D FOXVWHULQJ PHFKDQLVP WR
VHOHFWPRYHPHQWVIRUDQDUWLILFLDOGDQFHU
WKDWPLUURUZLWKVRPHGHYLDWLRQVWKRVHRI
DQ LQWHUDFWLQJ KXPDQ GDQFHU >@ 7KH
9LHZSRLQWV $, V\VWHP LPSOHPHQWV DQ
LQWHUDFWLRQEDVHGDXWKRULQJDSSURDFK IRU
WKH FUHDWLRQ RI V\QWKHWLF PRWLRQV WKDW
FRPELQHV LGHDV IURP FDVHEDVHG
OHDUQLQJDQGLPLWDWLYHOHDUQLQJ>@
:KHQ ZRUNLQJ ZLWK DXWRHQFRGHUV
QDYLJDWLQJWKHODWHQWVSDFH RI HQFRGLQJV
LV D SRSXODU PHWKRG IRU FUHDWLQJ QHZ
PRYHPHQW PDWHULDO 7KLV DSSURDFK KDV
EHHQ FKRVHQ ERWK E\ UHVHDUFKHUV
ZRUNLQJ ZLWK HQFRGLQJV RI SRVHV HJ
>@DQG UHVHDUFKHUVZRUNLQJZLWK
HQFRGLQJV RI SRVH VHTXHQFHV HJ >
@ 2QH RI WKH PDLQ GUDZEDFNV RI
QDYLJDWLRQ ODWHQW VSDFH SHUWDLQV WR WKH
GLIILFXOW\ RI REWDLQLQJ DQ XQGHUVWDQGLQJ
IRU WKH W\SLFDOO\ REVFXUH UHODWLRQVKLS
EHWZHHQ ODWHQW YHFWRUV DQG WKHLU
GHFRGLQJ6HYHUDOUHVHDUFKHUVKDYHWULHG
WR DGGUHVV WKLV LVVXH 2QH SRVVLEOH
DSSURDFKLVWRFRQGLWLRQWKHDXWRHQFRGHU
RQ KLJKHU OHYHO FRQWURO SDUDPHWHUV HJ
>@ $QRWKHU DSSURDFK LV WR H[WHQG DQ
DXWRHQFRGHU ZLWK D FRQWURO QHWZRUN WKDW
OHDUQVWRGLVDPELJXDWHODWHQWVSDFHHJ
>@
2.2.4 Digital Puppetry
7KH WHUP 'LJLWDO 3XSSHWU\ UHIHUV WR DQ\
VHW RI WHFKQLTXHV IRU LQWHUDFWLYHO\
FRQWUROOLQJ WKH EHKDYLRXUV RI FRPSXWHU
JHQHUDWHG FKDUDFWHUV 7KH EHKDYLRXUV
PLJKW QRW RQO\ LQYROYH WKH PRWLRQ RI WKH
DUWLILFLDO FKDUDFWHU EXW DOVR IRU LQVWDQFH
IDFLDO H[SUHVVLRQV DQG WKH XWWHULQJ RI
ZRUGV 'LJLWDO 3XSSHWU\ GUDZV LWV
LQVSLUDWLRQDO EDFNJURXQG IURP WUDGLWLRQDO
SXSSHWU\ DQG FRPELQHV LW ZLWK ´OLYH
DFWLRQ VWRS PRWLRQ DQLPDWLRQ JDPH
LQWHOOLJHQFH DQG RWKHU IRUPV LQWR DQ
HQWLUHO\QHZ PHGLXP´>@ $Q H[WHQVLYH
RYHUYLHZ RYHU WKH ILHOG LQFOXGLQJ D
UHFDSLWXODWLRQRIPDMRUGHYHORSPHQWVDQG
DSSOLFDWLRQV LQ SDUWLFXODU ZLWKLQ WKH
DQLPDWLRQLQGXVWU\LVSURYLGHGLQWKH3K'
WKHVLVE\/HLWH2UYDOKR>@
,Q WUDGLWLRQDO SXSSHWHHULQJ VXFK DV
PDULRQHWWHWKHDWUH D PHFKDQLFDOSXSSHW
LVVHW LQWR PRWLRQE\ PHDQVRID KXPDQ
SHUIRUPHU WKH SXSSHWHHU FRQWUROOLQJ D
VXEVHW RI WKH MRLQWV RI WKH SXSSHW 7KH
SXSSHW¶V UHPDLQLQJ MRLQWV WKHQ PRYH
DFFRUGLQJ WR SK\VLFDO ODZV 7KLV IRUP RI
LQWHUDFWLRQ WKDW GRHVQ¶W LQYROYH HQRXJK
GHJUHHV RI IUHHGRP '2) WR FRQWURO DOO
WKHMRLQWVRIDSXSSHWGLUHFWO\FRQVWLWXWHV
RQH RI WKH PDLQ WHFKQLFDO FKDOOHQJHV LQ
GLJLWDOSXSSHWU\7KHUH H[LVW DQXPEHURI
PHWKRGVWRFRQVWUXFWPRWLRQIRUDSXSSHW
WKDWLVXQGHUVSHFLILHGWKURXJKLQWHUDFWLRQ
7KHPRVWIUHTXHQWO\HPSOR\HGPHWKRGLV
WR JHQHUDWH PRWLRQ WKURXJK LQYHUVH
NLQHPDWLFV IRU WKRVH SXSSHW MRLQWV WKDW
DUH QRW GLUHFWO\ XQGHU WKH SXSSHWHHU¶V
FRQWURO HJ >@ $Q DOWHUQDWLYH LV WR
VLPXODWHDSXSSHW¶VSK\VLFDOG\QDPLFVWR
IXOO\ DUWLFXODWH LW $Q LQWHUHVWLQJ H[DPSOH
RI WKLV DSSURDFK KDV EHHQ LPSOHPHQWHG
LQ D V\VWHP WKDW UHFRQVWUXFWV KXPDQ
PRWLRQ IURP JURXQG UHDFWLRQ IRUFHV DQG
KDQGPRYHPHQWVRQO\>@$WKLUGRSWLRQ
LV WR SDUWLDOO\ GLVSRVH ZLWK GLUHFW DQG
FRQWLQXRXV FRQWURO RI SXSSHW MRLQWV DQG
LQVWHDG VHOHFW WKH SXSSHW¶V SRVHV DQG
DFWLRQVEDVHGRQFRQWH[W $ FRPELQDWLRQ
RI SK\VLFV VLPXODWLRQ DQG EHKDYLRXU
VHOHFWLRQKDVEHHQSURSRVHGE\,VKHJDNL
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


DQG FROOHDJXHV IRU FRQWUROOLQJ WKH
EHKDYLRXU RI D FKDUDFWHU LQ D YLUWXDO
HQYLURQPHQW >@ +HUH WKH VHOHFWLRQ RI
EHKDYLRXUVLVEDVHGRQDFRPELQDWLRQRI
UHFRJQLVLQJDXVHU¶VLQWHQWDQGVLPXODWLQJ
RQO\WKRVHDFWLRQVWKDWDUHDSSURSULDWHLQ
WKH FXUUHQW VLWXDWLRQ ZLWKLQ WKH YLUWXDO
HQYLURQPHQW 6HYHUDO UHVHDUFKHUV KDYH
H[SORUHG WKH XVH RI PRWLRQ FDSWXUH
UHFRUGLQJVLQFRPELQDWLRQZLWKLQWHUDFWLRQ
PHFKDQLVPV IRU JHQHUDWLQJ PRWLRQV IRU
GLJLWDO FKDUDFWHUV WKDW DUH GHULYHG IURP
WKHVH UHFRUGLQJV 2QH H[DPSOH XVHV
PDFKLQH OHDUQLQJ WR UHFRJQLVH D XVHU¶V
LQGLYLGXDOLVHG KDQG SRVHV DQG WKHQ
VHOHFW D FKDUDFWHU DQLPDWLRQ IURP D
GDWDEDVH RI KXPDQ ORFRPRWLRQ
UHFRUGLQJV>@$QRWKHU H[DPSOH XVHVD
JHQHUDWLYH UHFXUUHQW PDFKLQHOHDUQLQJ
PRGHO WKDW KDV EHHQ WUDLQHG RQ PRWLRQ
FDSWXUH UHFRUGLQJV IRU V\QWKHVLVLQJ
PRWLRQV RI D GLJLWDO FKDUDFWHU ZKLOH LW
LQWHUDFWVZLWKDKXPDQSOD\HU>@
3. Implementation
7KH V\VWHP HPSOR\HG LQ WKLV UHVHDUFK
XVHV D JHQHUDWLYH PDFKLQHOHDUQLQJ
PRGHO IRU V\QWKHVLVLQJ VKRUW SRVH
VHTXHQFHV LQ FRPELQDWLRQ ZLWK D
EOHQGLQJDQGVHTXHQFLQJPHFKDQLVPIRU
FRQFDWHQDWLQJWKHJHQHUDWHGVHTXHQFHV
7KLV V\VWHP ZLWK WKH QDPH *UDQXODU
'DQFH KDV EHHQ GHYHORSHG E\ WKH ILUVW
DXWKRUDQGLVGHVFULEHGLQVRPHGHWDLOLQ
D SUHYLRXV SXEOLFDWLRQ >@ 7KH PDLQ
DUFKLWHFWXUH DQG RSHUDWLRQ RI *UDQXODU
'DQFHUHPDLQVXQFKDQJHGDQGLV EULHIO\
VXPPDULVHG LQ WKLV DUWLFOH :KDW KDV
EHHQ DGGHG LQ WKH PHDQWLPH LV D
PHFKDQLVPWRFUHDWHV\QWKHWLFPRWLRQVLQ
ZKLFK RQH RU VHYHUDO MRLQWV FDQ EH
LQWHUDFWLYHO\ FRQWUROOHG ZKLOH WKH
UHPDLQLQJMRLQWVUHPDLQXQGHUWKHFRQWURO
RI WKH PDFKLQHOHDUQLQJ PRGHO 7KLV
FRPELQDWLRQ RI LQWHUDFWLYH FRQWURO DQG
PDFKLQHOHDUQLQJ RIIHUV WKHSRVVLELOLW\WR
FUHDWH DQ DUWLILFLDO GDQFHU WKDW SDUWLDOO\
PLUURUV WKH DFWLYLWLHV RI D KXPDQ
SHUIRUPHU EXW DW WKH VDPH WLPH DOVR
H[KLELWV DXWRQRPRXV PRWLRQV WKDW
SUHVHUYHVRPHRIWKHVW\OLVWLFXQLTXHQHVV
RIWKHRULJLQDOPRWLRQFDSWXUHUHFRUGLQJV
3.1 Granular Dance
7KH PDFKLQHOHDUQLQJ SDUW RI *UDQXODU
'DQFH FRQVLVWV RI DQ DGYHUVDULDO
DXWRHQFRGHU WKDW KDV EHHQ WUDLQHG RQ
PRWLRQ FDSWXUH UHFRUGLQJV RI D
SURIHVVLRQDO GDQFHU ZKR IUHHO\
LPSURYLVHG WR PXVLF 7KH PRGHO¶V
DUFKLWHFWXUH FRQVLVWV RI WKUHH QHXUDO
QHWZRUNV DQ HQFRGHU GHFRGHU DQG
GLVFULPLQDWRU ILJXUH  OHIW VLGH 7KH
HQFRGHU DQG GHFRGHU SDUWV DUH
DXWRUHJUHVVLYH 7KH HQFRGHU WDNHV D
VHTXHQFH RI SRVHV DV LQSXW DQG
JHQHUDWHV D ORZ GLPHQVLRQDO
UHSUHVHQWDWLRQ RI WKLV VHTXHQFH LQ WKH
IRUP RI D ODWHQW YHFWRU DV RXWSXW 7KH
GHFRGHU WDNHV ODWHQW YHFWRU DV LQSXW D
DQGUHFRQVWUXFWVDVHTXHQFHRISRVHVDV
RXWSXW 7KH GLVFULPLQDWRU WDNHV D ODWHQW
YHFWRUDVLQSXWDQGSURGXFHVDVRXWSXWD
ELQDU\YDOXHLQGLFDWLQJLWVDVVHVVPHQWRI
ZKHWKHU WKH YHFWRU IROORZV D *DXVVLDQ
GLVWULEXWLRQ RU QRW 7KLV PHFKDQLVP
HQVXUHV WKDW WKH VSDFH RI ODWHQW YHFWRUV
LVIUHHRIJDSVDQGWKDWGLVWDQFHVZLWKLQLW
UHSUHVHQWDPHDVXUHRIVLPLODULW\7KLVLQ
WXUQHQVXUHVWKDWDUELWUDULO\FKRVHQODWHQW
YHFWRUV FDQ EH GHFRGHG LQWR PHDQLQJIXO
SRVHVHTXHQFHV
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH

)LJXUH  *UDQXODU 'DQFH 6\VWHP
$UFKLWHFWXUH6KRZQ RQ WKHOHIW VLGH LV D
VFKHPDWLFUHSUHVHQWDWLRQRIWKHPDFKLQH
OHDUQLQJ PRGHO GHSLFWLQJ LWV WKUHH QHXUDO
QHWZRUNV DQ HQFRGHU D GHFRGHU DQG D
GLVFULPLQDWRU 7KH WUDSH]RLG VKDSHV RI
WKH HQFRGHU DQG GHFRGHU LQGLFDWH WKH
GLPHQVLRQ UHGXFWLRQ DQG H[SDQVLRQ WKDW
LVFRQGXFWHGE\WKHP7KHFLUFXODUUHJLRQ
UHSUHVHQWV WKH ODWHQW VSDFH RI SRVH
VHTXHQFHHQFRGLQJV6KRZQRQWKHULJKW
VLGH LV WKH SRVH VHTXHQFH EOHQGLQJ
PHFKDQLVP WKDW PLPLFV JUDQXODU DQG
FRQFDWHQDWLYH V\QWKHVLV WHFKQLTXHV
6HTXHQFHV DUH REWDLQHG E\ GHFRGLQJ
VDPSOHG ODWHQW VSDFH YHFWRUV 0XOWLSOH
GHFRGHG VHTXHQFHV DUH EOHQGHG LQWR
ORQJHU VHTXHQFHV XVLQJ RYHUODSSLQJ
ZLQGRZVZKLFK DUH GHSLFWHGKHUHDVEHOO
VKDSHGFXUYHV
7KH VHTXHQFH EOHQGLQJ PHFKDQLVP
VHUYHVWKH SXUSRVH RI FRQFDWHQDWLQJWKH
VKRUW SRVH VHTXHQFHV JHQHUDWHG E\ WKH
DXWRHQFRGHU LQWR ORQJHU VHTXHQFHV
ILJXUH  ULJKW VLGH 7KH PHFKDQLVP
GUDZV LQVSLUDWLRQ IURP PHWKRGV LQ
FRPSXWHU PXVLF WKDW FRPELQH VKRUW
VRXQG IUDJPHQWV WR JHQHUDWH ORQJHU
VRXQGV *UDQXODU 6\QWKHVLV >@ DQG
&RQFDWHQDWLYH 6\QWKHVLV >@ 6LQFH
SRVHVDUHUHSUHVHQWHGE\ MRLQW DQJOHV LQ
WKH IRUP RI XQLW TXDWHUQLRQV WKHLU
EOHQGLQJ LV EDVHG RQ VSKHULFDO OLQHDU
LQWHUSRODWLRQ 6/(53 >@ 6RXUFH
VHTXHQFHV DUH EOHQGHG RQH DIWHU WKH
RWKHU ZLWK D WDUJHW VHTXHQFH ZLWK ZKLFK
WKH\ DUH DOLJQHG DW VXFFHVVLYHO\
LQFUHDVLQJ SRVLWLRQV 7KH DPRXQW RI
EOHQGLQJ EHWZHHQ VRXUFH DQG WDUJHW LV
FRQWUROOHGE\DZLQGRZ IXQFWLRQ+DQQLQJ
LQWKLVFDVH
3.2 Iterative Autoencoding and
Interaction
7KHRULJLQDO*UDQXODU'DQFH V\VWHPKDV
EHHQ PRGLILHG IRU UHDOWLPH DSSOLFDWLRQ
ZKLFK RSHQV XS WKH SRVVLELOLW\ RI
LQWHUDFWLYHO\FRQWUROOLQJVRPHRUDOORIWKH
MRLQWRULHQWDWLRQVLQ DSRVHVHTXHQFH$V
SDUW RI WKLV DGDSWDWLRQ WKH V\VWHP KDV
EHHQ SRUWHG IURP DQ LPSOHPHQWDWLRQ
EDVHG RQ WKH 7HQVRUIORZ PDFKLQH
OHDUQLQJ IUDPHZRUN WR RQH XVLQJ WKH
3\7RUFK IUDPHZRUN7KLVFKDQJHPDGH
LWSRVVLEOHWRLQWHJUDWHWKHWUDLQHGQHXUDO
QHWZRUNV LQWR D VRIWZDUH VXLWDEOH IRU
LQWHUDFWLYH UHDOWLPH DSSOLFDWLRQV 7KLV
VRIWZDUH ZDV ZULWWHQ LQ & XVLQJ WKH
RSHQ)UDPHZRUNV FUHDWLYH FRGLQJ
HQYLURQPHQW$OVRIRUUHDOWLPHSXUSRVHV
D QHXUDO QHWZRUN KDV EHHQ FKRVHQ WKDW
RSHUDWHVRQVHTXHQFHVWKDW DUH SRVHV
ORQJ DV RSSRVHG WR  SRVHV XVHG LQ
VRPHRIWKHH[SHULPHQWVGHVFULEHGLQWKH
RULJLQDO SXEOLFDWLRQ %\ RSHUDWLQJ RQ
VXFK VKRUW VHTXHQFHV WKH PRGHO FDQ
V\QWKHVLVH QHZ PRWLRQV IUHTXHQWO\
HQRXJK WR EH VXLWDEOH IRU LQWHUDFWLYH
DSSOLFDWLRQV
7KH PDLQ QRYHOW\ LQWURGXFHG LQ WKH QHZ
LPSOHPHQWDWLRQLQYROYHV WKH FRPELQDWLRQ
RI DQ LWHUDWLYH DXWRHQFRGLQJ PHFKDQLVP
ZLWK LQWHUDFWLYH FRQWURO RI VRPH RI WKH
MRLQW RULHQWDWLRQV ,Q LWHUDWLYH
DXWRHQFRGLQJ WKH RXWSXW RI WKH GHFRGHU
7HQVRUIORZKWWSVZZZWHQVRUIORZRUJ
3\7RUFKKWWSVS\WRUFKRUJ
RSHQ)UDPHZRUNVKWWSVRSHQIUDPHZRUNVFF
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


LV UHSHDWHGO\ IHHG EDFN DV LQSXW IRU WKH
HQFRGHU (DFK RI WKHVH LWHUDWLRQV VWDUWV
ZLWK D SRVH VHTXHQFH WKDW LV REWDLQHG
IURP SOD\EDFN RI WKH RULJLQDO PRWLRQ
FDSWXUHUHFRUGLQJ7KLVSRVHVHTXHQFHLV
WKHQ JUDGXDOO\ WUDQVIRUPHG WKURXJK WKH
UHSHDWHG HQFRGLQJ DQG GHFRGLQJ VWHSV
XQWLOLWFRQYHUJHVLQWR DVHTXHQFHWKDWLV
FRQVLGHUHG E\ WKH DXWRHQFRGHU WR EH
VWDWLVWLFDOO\ PRVW UHSUHVHQWDWLYH RI WKH
RULJLQDOO\ UHFRUGHG PDWHULDO ,QWHUDFWLYLW\
LQWHUIHUHVZLWK WKLVFRQYHUJHQFH SURFHVV
E\ LW VHWWLQJ WKH URWDWLRQV RI WKRVH MRLQWV
WKDW DUH LQWHUDFWLYHO\ FRQWUROOHG DQG
SUHYHQWLQJ WKHP IURP EHLQJ PRGLILHG E\
WKH DXWRHQFRGHU 7KH DXWRHQFRGHU LV
IRUFHGWRVKLIWWKHFRQYHUJHQFHWRZDUGVD
SRVH VHTXHQFH WKDW UHVSHFWV WKH MRLQW
URWDWLRQV VSHFLILHG WKURXJK LQWHUDFWLRQ
DQGLVVWDWLVWLFDOO\UHSUHVHQWDWLYH
7KH QXPEHU RI LWHUDWLRQV XVHG IRU
FRQYHUJHQFH YLD DXWRHQFRGLQJ DQG WKH
QXPEHU RI LWHUDWLRQV GXULQJ ZKLFK WKH
MRLQWV DUH LQWHUDFWLYHO\ FRQWUROOHG GR QRW
QHFHVVDULO\KDYHWR EH WKHVDPH,IWKH\
DUHWKHVDPHWKHMRLQWURWDWLRQVVSHFLILHG
WKURXJK LQWHUDFWLRQ ZLOO UHPDLQ UHPRYHG
IURP WKH LQIOXHQFH RI WKH DXWRHQFRGHU
XQWLOWKHHQGRIFRQYHUJHQFHDQGDUHIXOO\
YLVLEOHLQ WKH ILQDOSRVH VHTXHQFH %XWLI
WKH\ DUH QRW WKH LQWHUDFWLYHO\ FRQWUROOHG
MRLQW URWDWLRQV ZLOO RQO\ EH NHSW UHPRYHG
IURPWKH LQIOXHQFH RIWKHDXWRHQFRGHU DW
WKH EHJLQQLQJ RI FRQYHUJHQFH DQG
DIWHUZDUGV FKDQJH DV UHVXOW RI WKH
DXWRHQFRGLQJ SURFHVV $FFRUGLQJO\ E\
YDU\LQJ WKHVH LWHUDWLRQ QXPEHUV WKH
DPRXQWRILQIOXHQFHLQWHUDFWLRQH[HUWVRQ
WKH DXWRHQFRGLQJ SURFHVV FDQ EH
PRGLILHG )RU SXUSRVHV RI EUHYLW\ WKH
IROORZLQJ WZR WHUPV ZLOO EH XVHG IURP
QRZRQWRUHIHUWRWKHWZRGLIIHUHQWW\SHV
RI LWHUDWLRQV FRGLQJ LWHUDWLRQV VWDQGV IRU
WKH QXPEHU RI LWHUDWLRQV RI WKH
DXWRHQFRGHULQWHUDFWLRQLWHUDWLRQVVWDQGV
IRU WKH QXPEHU RI LWHUDWLRQV WKH
RULHQWDWLRQVRI WKHMRLQWV DUHVHW WKURXJK
LQWHUDFWLRQ ,W LV ZRUWKZKLOH WR PHQWLRQ
WKDWWKHPHWKRGFKRVHQKHUHLVUHODWHGWR
WZR GLVWLQFW RSHUDWLRQDO SULQFLSOHV RI
DXWRHQFRGHUV2QHSULQFLSOHLVEDVHGRQ
WKH GHQRLVLQJ FDSDELOLW\ RI
DXWRHQFRGHUV 6LQFH DXWRHQFRGHUV
UHSUHVHQW KLJK GLPHQVLRQDO GDWD DV ORZ
GLPHQVLRQDO HQFRGLQJV WKH\ DUH IRUFHG
WR LJQRUH LQVWDQWLDWLRQV WKDW DUH
VWDWLVWLFDOO\ UDUH RU H[KLELW XQV\VWHPDWLF
YDULDQFH 5HSHDWHGO\ DSSO\LQJ DQ
DXWRHQFRGHUOHDGVWRWKHUHPRYDORIVXFK
LQVWDQWLDWLRQV ZLWK WKH QXPEHU RI
LWHUDWLRQV FRQWUROOLQJ WKH H[WHQW RI WKLV
UHPRYDO7KHRWKHUSULQFLSOHLVEDVHGRQ
WKH FRQGLWLRQLQJ RI DXWRHQFRGHUV
$XWRHQFRGHUVFDQEHFRQGLWLRQHGRQWKH
W\SHRIGDWDWKH\JHQHUDWHE\DGGLQJRQH
RU VHYHUDO FRQGLWLRQLQJ YDULDEOHV WR WKH
LQSXW RI ERWK WKH HQFRGHU DQG GHFRGHU
%\ IL[LQJ VRPH RI WKH MRLQW URWDWLRQV
GXULQJ DXWRHQFRGLQJ WKHVH URWDWLRQV
EHFRPH FRQGLWLRQLQJ YDULDEOHV IRU WKH
HQFRGHU %XW LQ WKH PHWKRG HPSOR\HG
KHUH WKHVH URWDWLRQV DUH RQO\ SURYLGHG
DV FRQGLWLRQLQJ YDULDEOHV WR WKH HQFRGHU
DQG QRW WR WKH GHFRGHU 7KHUHIRUH WKH
FXUUHQW PHWKRG LV QRW IXOO\ HTXLYDOHQW WR
DXWRHQFRGHUFRQGLWLRQLQJ
4. Results and Discussion
3ULRU WR WKH H[SHULPHQWV GHVFULEHG
EHORZWKHDXWRHQFRGHUKDVEHHQWUDLQHG
RQ WKH VDPH PRWLRQ FDSWXUH GDWD DQG
ZLWK WKH VDPH WUDLQLQJ FRQILJXUDWLRQ DV
WKHRULJLQDO*UDQXODU 'DQFH V\VWHP 7KH
UHFRUGHGVXEMHFWLVDSURIHVVLRQDOGDQFHU
ZKRZDVIUHHO\LPSURYLVLQJWRPXVLF
6HYHUDO H[SHULPHQWV KDYH EHHQ
FRQGXFWHG WKDW GLIIHU IURP HDFK RWKHU
ZLWKUHVSHFW WRWKHH[FHUSWVFKRVHQIURP
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


WKHPRWLRQFDSWXUHUHFRUGLQJWKHPHWKRG
IRU SURYLGLQJ LQWHUDFWLYH FRQWURO DQG WKH
QXPEHU RI FRGLQJ LWHUDWLRQV DQG
LQWHUDFWLRQLWHUDWLRQV
)LYH H[FHUSWV IURP WKH PRWLRQ FDSWXUH
UHFRUGLQJV KDYH EHHQ VHOHFWHG (DFK RI
WKHVHH[FHUSWVLVIUDPHVORQJZKLFK
FRUUHVSRQGVWRDUHFRUGLQJGXUDWLRQ RI 
VHFRQGV([FHUSW VLWWLQJ SRVHZLWK QR
ERG\ PRWLRQ ([FHUSW  VORZ WRUVR
EHQGLQJ ZLWK OLWWOH DUP DQG OHJ PRWLRQ
([FHUSW DUPDQG OHJPRWLRQ ZLWKOLWWOH
FRRUGLQDWLRQ ([FHUSW  DUP DQG OHJ
PRWLRQ ZLWK VWURQJ FRRUGLQDWLRQ
([FHUSW  IXOO ERG\ PRWLRQ ZLWK SDUWLDO
FRRUGLQDWLRQ )RU LOOXVWUDWLYH SXUSRVHV
DOO H[FHUSWV DUH GHSLFWHG DV ILOPVWULSV
ILJXUH
$VDILUVWVHWRIH[SHULPHQWVWKHQXPEHU
RIFRGLQJLWHUDWLRQVZDVYDULHGEHWZHHQ
DQG  ZLWK QR LQWHUDFWLYH FRQWURO RI
MRLQWV 7KHVH H[SHULPHQWV VHUYHG WKH
SXUSRVHRIHYDOXDWLQJKRZDXWRHQFRGLQJ
LWVHOI FRQYHUJHV WKH GLIIHUHQW H[FHSWV
7KH UHVXOWV RI WKHVH H[SHULPHQW DUH
DYDLODEOH RQOLQH DV YLGHR
([FHUSW0RFDS
KWWSVYLPHRFRPIDFF
([FHUSW0RFDS
KWWSVYLPHRFRPH
([FHUSW0RFDS
KWWSVYLPHRFRPDDHFF
([FHUSW0RFDS
KWWSVYLPHRFRPIDHII
([FHUSW0RFDS
KWWSVYLPHRFRPIDH
([FHUSW&RGLQJ,WHUDWLRQV
KWWSVYLPHRFRPDHI
([FHUSW&RGLQJ,WHUDWLRQV
KWWSVYLPHRFRPFEGH
([FHUSW&RGLQJ,WHUDWLRQV
KWWSVYLPHRFRP
([FHUSW&RGLQJ,WHUDWLRQV
KWWSVYLPHRFRPEFGD
6RPH RI WKH UHVXOWV DUH GHSLFWHG DV
ILOPVWULSVILJXUH

)LJXUH  ([FHUSWV RI 0RWLRQ &DSWXUH
5HFRUGLQJV )URP WRS WR ERWWRP WKH
ILOPVWULSVGHSLFWH[FHUSWVWR

)LJXUH 9DULDWLRQV LQ &RGLQJ ,WHUDWLRQV
)URPWRSWRERWWRPWKHILOPVWULSVGHSLFW
H[FHUSW  FRGLQJ LWHUDWLRQV  H[FHUSW 
FRGLQJ LWHUDWLRQV  H[FHUSW  FRGLQJ
LWHUDWLRQV  H[FHUSW  FRGLQJ LWHUDWLRQV

7KH UHVXOWV RI WKHVH FRGLQJ LWHUDWLRQ
H[SHULPHQWV FDQ EH VXPPDULVHG DV
IROORZV :KHQ FRQGXFWLQJ D VLQJOH
LWHUDWLRQ WKH DXWRHQFRGHU IDLWKIXOO\
UHFRQVWUXFWVWKHRULJLQDOPRWLRQZLWKOLWWOH
TXDOLWDWLYH GLIIHUHQFH $V WKH QXPEHU RI
LWHUDWLRQV LQFUHDVHV WKH UHFRQVWUXFWHG
PRWLRQEHJLQVWRGHYLDWHIURPWKHRULJLQDO
PRWLRQ 7KH RQO\ H[FHSWLRQ LV H[FHUSW 
ZKLFK FRQVLVWV RI D PRVWO\ VWDWLF SRVH
WKDW LV SURSHUO\ UHFRQVWUXFWHG DOVR DW
KLJKHU QXPEHUV RI FRGLQJ LWHUDWLRQV
6RPHRIWKHFKDQJHVLQWURGXFHGWKURXJK
PXOWLSOH FRGLQJ LWHUDWLRQV RFFXU RQO\ IRU
VRPH H[FHUSWV ZKLOH RWKHUV DUH PRUH
FRPPRQ )RU H[FHUSW  DQ LQFUHDVH LQ
([FHUSW&RGLQJ,WHUDWLRQV
KWWSVYLPHRFRPGI
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


FRGLQJ LWHUDWLRQV UHVXOWV LQ D URWDWLRQ RI
WKH VNHOHWRQ WKDW PDNHV LW IURQW IDFLQJ
HJ URZV  DQG  LQ ILJXUH  )RU DOO
H[FHUSWVH[FHSWQXPEHUDQLQFUHDVHLQ
FRGLQJ LWHUDWLRQV FDXVHV WKH RULJLQDOO\
VPRRWK PRWLRQ WR EHFRPH LQWHUPLWWHQW
ZLWK WKH VNHOHWRQ SDXVLQJ RQ D SRVH
EHIRUHMXPSLQJWR WKHQH[WSRVH,QPRVW
FDVHV WKH SRVHV WKDW DUH VXVWDLQHG
RFFXULQWKHRULJLQDOH[FHUSWHJURZLQ
ILJXUH ,QDIHZFDVHVH[FHUSWVDQG
WKHVNHOHWRQDOVR KDOWV RQ SRVHVWKDW
GRQRW RFFXU LQWKHRULJLQDO H[FHUSW HJ
IUDPHLQURZVDQGLQILJXUH
,QDVHFRQG VHW RI H[SHULPHQWVDVLQJOH
MRLQW WKH OHIW VKRXOGHU ZDV LQWHUDFWLYHO\
FRQWUROOHG7KHGHFLVLRQWRLQWHUDFWZLWKD
VLQJOH MRLQW RQO\ VHUYHG WKH SXUSRVH RI
VLPSOLI\LQJWKHHYDOXDWLRQRIWKHLQWHUSOD\
EHWZHHQ LQWHUDFWLRQ DQG DXWRHQFRGLQJ
$OVR LQWHUDFWLQJ ZLWK D VLQJOH MRLQW
FRQVWLWXWHVWKHPRVWH[WUHPHVFHQDULRIRU
WHVWLQJ WKH VXLWDELOLW\ RI WKH FKRVHQ
DSSURDFK IRU JHQHUDWLQJ V\QWKHWLF
PRWLRQV WKURXJK XQGHUVSHFLILHG
LQWHUDFWLRQ
7KH ILUVW LQWHUDFWLRQ H[SHULPHQW GLGQ¶W
LQYROYHDQ\LQWHUDFWLRQDWDOO 5DWKHU WKH
RULHQWDWLRQ RI WKH MRLQW VHOHFWHG IRU
LQWHUDFWLRQ ZDV HLWKHU IXOO\ RU SDUWLDOO\
UHPRYHG IURP EHLQJ PRGLILHG E\ WKH
DXWRHQFRGHU EXW RWKHUZLVH OHIW
XQFKDQJHG7KLVZDVGRQHE\LQFUHDVLQJ
WKHQXPEHURILQWHUDFWLRQLWHUDWLRQVWRWKH
VDPH YDOXH RU KDOI RI WKH YDOXH RI WKH
QXPEHURIFRGLQJLWHUDWLRQVUHVSHFWLYHO\
7KHVH H[SHULPHQWV ZHUH FRQGXFWHG IRU
HDFK H[FHUSW ZLWK WKH IROORZLQJ
YDULDWLRQV FRGLQJ LWHUDWLRQV  DQG
LQWHUDFWLRQ LWHUDWLRQVFRGLQJ LWHUDWLRQV
 DQG LQWHUDFWLRQ LWHUDWLRQV  FRGLQJ
LWHUDWLRQV DQGLQWHUDFWLRQ LWHUDWLRQV
7KH UHVXOWV RI WKHVH H[SHULPHQWV DUH
DYDLODEOHRQOLQHDVYLGHRV,Q
FDVH RI H[FHUSW  WKH UHVXOWV DUH DOVR
GHSLFWHGDVILOPVWULSVILJXUH

)LJXUH  ,QWHUDFWLYHO\ &RQWUROOHG -RLQW
ZLWK )L[HG 2ULHQWDWLRQ )URP WRS WR
ERWWRP WKH ILOPVWULSV GHSLFW H[FHUSW 
FRGLQJ LWHUDWLRQV  LQWHUDFWLRQ LWHUDWLRQV
 H[FHUSW  FRGLQJ LWHUDWLRQV 
LQWHUDFWLRQ LWHUDWLRQV  H[FHUSW  FRGLQJ
LWHUDWLRQVLQWHUDFWLRQLWHUDWLRQV

)LJXUH  ,QWHUDFWLYHO\ &RQWUROOHG -RLQW
ZLWK &KDQJLQJ 2ULHQWDWLRQ )URP WRS WR
ERWWRP WKH ILOPVWULSV GHSLFW H[FHUSW 
FRGLQJLWHUDWLRQV  LQWHUDFWLRQLWHUDWLRQV
F\FOH GXUDWLRQ  VHFH[FHUSW FRGLQJ
LWHUDWLRQV  LQWHUDFWLRQ LWHUDWLRQV 
F\FOH GXUDWLRQ  VHF H[FHUSW  FRGLQJ
LWHUDWLRQVLQWHUDFWLRQLWHUDWLRQV F\FOH
GXUDWLRQ  VHF H[FHUSW  FRGLQJ
LWHUDWLRQV  LQWHUDFWLRQ LWHUDWLRQV 
F\FOHGXUDWLRQVHF
([FHUSW)L[HG2ULHQWDWLRQ
KWWSVYLPHRFRP
([FHUSW)L[HG2ULHQWDWLRQ
KWWSVYLPHRFRPHGEDGI
([FHUSW)L[HG2ULHQWDWLRQ
KWWSVYLPHRFRPEDDI
([FHUSW)L[HG2ULHQWDWLRQ
KWWSVYLPHRFRPIFI
([FHUSW)L[HG2ULHQWDWLRQ
KWWSVYLPHRFRPFEFFF
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


7KH UHVXOWV FDQ EH VXPPDULVHG DV
IROORZV )RU DOO H[FHUSWV WKH IL[HG
RULHQWDWLRQ RI WKH VLQJOH MRLQW LV RQO\
FOHDUO\ SUHVHQW LQ WKH JHQHUDWHG PRWLRQ
ZKHQ WKH QXPEHU RI FRQWURO DQG
LQWHUDFWLRQLWHUDWLRQVDUHWKHVDPH,IWKH
QXPEHU RI LQWHUDFWLRQ LWHUDWLRQVLVORZHU
WKDWWKH QXPEHU RIFRQWURO LWHUDWLRQV WKH
LQLWLDOO\ IL[HG RULHQWDWLRQ LV TXLFNO\
FKDQJHG E\ WKH DXWRHQFRGLQJ SURFHVV
7KHLQIOXHQFHRIWKHIL[HGMRLQWRULHQWDWLRQ
RQ WKH UHPDLQLQJ MRLQWV LV YHU\ VPDOO
ZKHQ WKH QXPEHU RI FRGLQJ DQG
LQWHUDFWLRQLWHUDWLRQVLVRQH,QWKLVFDVH
WKH RULJLQDO PRWLRQ LQ HDFK RI WKH
H[FHUSWV UHPDLQV ODUJHO\ LQWDFW $V WKH
QXPEHU RI LWHUDWLRQV LQFUHDVHV WKH
GHYLDWLRQ IURP WKH RULJLQDO PRWLRQ DOVR
LQFUHDVHV ZLWK H[FHUSW  EHLQJ WKH RQO\
H[FHSWLRQ ,Q FDVH RI H[FHUSW  WKH
SUHVHQFHRILQWHUPLWWHQWSRVHVLQFUHDVHV
ZLWK PRVW RI WKHVH SRVHV QRW RULJLQDOO\
SUHVHQW LQ WKH H[FHUSW URZV  DQG  LQ
ILJXUH  ,Q FDVH RI H[FHUSW  DQG 
LQWHUPLWWHQWSRVHVDSSHDURQO\EULHIO\DQG
DUHTXLFNO\VZLWFKHG EHWZHHQ,QFDVHRI
H[FHUSW  VHYHUDO EULHI DQG RQH ORQJ
LQWHUPLWWHQW SRVH DUH SUHVHQW 7KH EULHI
SRVHV DSSHDU GXULQJ PRPHQWV LQ WKH
RULJLQDOH[FHUSWWKDWFRQWDLQIUHTXHQWDUP
PRWLRQV7KHORQJSRVHDSSHDUVWRZDUGV
WKHHQG RI WKHRULJLQDO H[FHUSW ZKHQWKH
DUPVEDUHO\PRYH
,QWKHVHFRQGLQWHUDFWLRQH[SHULPHQWWKH
RULHQWDWLRQRIDVLQJOHMRLQWZDVFKDQJHG
E\ URWDWLQJ WKH MRLQW DW FRQVWDQW YHORFLW\
DQGDURXQGDFRQVWDQWURWDWLRQD[LVRYHU
VHYHUDO IXOO UHYROXWLRQV 7ZR YHORFLWLHV
ZHUH FKRVHQ IRU WKH URWDWLRQ RQH
FRPSOHWLQJD UHYROXWLRQ LQ VHFRQGDQG
WKH RWKHU LQ  VHFRQGV 7KHVH
H[SHULPHQWVZHUH FRQGXFWHG IRUHDFKRI
WKH H[FHUSWV ZLWK WKH VDPH YDULDWLRQV LQ
WKH QXPEHU RI FRGLQJ DQG LQWHUDFWLRQ
LWHUDWLRQVDVLQWKHSUHYLRXVH[SHULPHQWV
7KH UHVXOWV RI WKHVH H[SHULPHQWV DUH
DYDLODEOHRQOLQH DV YLGHRV
,QFDVHRIH[FHUSWWKHUHVXOWV
DUHDOVRGHSLFWHGDVILOPVWULSVILJXUH
7KH UHVXOWV FDQ EH VXPPDULVHG DV
IROORZV )RU DOO H[FHUSWV WKH URWDWLQJ
LQWHUDFWLRQ MRLQW LV RQO\ FOHDUO\
SHUFHLYDEOH LQ WKH JHQHUDWHG PRWLRQ
ZKHQ WKH QXPEHU RI FRQWURO DQG
LQWHUDFWLRQ LWHUDWLRQVDUHWKHVDPH$OVR
IRU DOO H[FHUSWV WKH LQIOXHQFH RI WKH
URWDWLQJLQWHUDFWLRQMRLQWRQWKHUHPDLQLQJ
VNHOHWRQ MRLQWV LV VWURQJHVW ZKHQ WKH
URWDWLRQOHDGVWRDMRLQWRULHQWDWLRQWKDWLV
PDUNHGO\GLIIHUHQWIURPWKHRQHRULJLQDOO\
SUHVHQW LQ WKH H[FHUSW 7KLV HIIHFW LV
SDUWLFXODUO\ SURQRXQFHG ZKHQ WKH
RULHQWDWLRQ RI WKH MRLQW LV XQUHDOLVWLF )RU
DOPRVW DOO H[FHUSWV WKH YHORFLW\ RI WKH
MRLQW URWDWLRQ DIIHFWV WKH SDFLQJ RI WKH
HQWLUH VNHOHWRQ PRWLRQ 7KLV LV HYHQ WKH
FDVH IRU H[FHUSW  LQ ZKLFK WKH MRLQW
URWDWLRQDGGVPRWLRQWRWKHIRUPHUO\VWDWLF
VNHOHWRQ SRVH 7KH HIIHFW RI PRWLRQ
([FHUSW&KDQJLQJ2ULHQWDWLRQ6HFRQG
KWWSVYLPHRFRPIHHFGGH
([FHUSW&KDQJLQJ2ULHQWDWLRQ6HFRQGV
KWWSVYLPHRFRPIG
([FHUSW&KDQJLQJ2ULHQWDWLRQ6HFRQG
KWWSVYLPHRFRPFDE
([FHUSW&KDQJLQJ2ULHQWDWLRQ6HFRQGV
KWWSVYLPHRFRPGGHIIE
([FHUSW&KDQJLQJ2ULHQWDWLRQ6HFRQG
KWWSVYLPHRFRPDEEH
([FHUSW&KDQJLQJ2ULHQWDWLRQ6HFRQGV
KWWSVYLPHRFRPEDI
([FHUSW&KDQJLQJ2ULHQWDWLRQ6HFRQG
KWWSVYLPHRFRPIHEE
([FHUSW&KDQJLQJ2ULHQWDWLRQ6HFRQGV
KWWSVYLPHRFRPE
([FHUSW&KDQJLQJ2ULHQWDWLRQ6HFRQG
KWWSVYLPHRFRPEEFEGE
([FHUSW&KDQJLQJ2ULHQWDWLRQ6HFRQGV
KWWSVYLPHRFRPFGHGDI
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


SDFLQJLVZHDNHVWZKHQWKHRULHQWDWLRQRI
WKH URWDWLQJ MRLQW LV VLPLODU WR WKDW RI WKH
RULJLQDOH[FHUSWDQGLWVLVVWURQJHVWLILWLV
GLVVLPLODU ,Q FDVH RI H[FHUSW  WKH
DPSOLWXGHRIWKHLQGXFHGPRWLRQLVODUJHU
IRUVORZMRLQWURWDWLRQVWKDQIRUIDVWRQHV
([FHUSW  LV DQ H[FHSWLRQ LQ WKDW WKH
SDFLQJDQG RXWOLQH RIWKHRULJLQDO PRWLRQ
LVODUJHO\XQDIIHFWHGE\WKHURWDWLQJMRLQW
)RU WKH ODVW LQWHUDFWLRQ H[SHULPHQW WKH
RULHQWDWLRQ RI WKH LQWHUDFWLRQ MRLQW ZDV
FRQWUROOHG ZLWK D ZHDUDEOH VHQVRU WKDW
ZDV DWWDFKHG WR WKH KDQG RI RQH RI WKH
DXWKRUV 7KH VHQVRU LV DQ LQHUWLDO
PHDVXUHPHQW XQLW ,08 ZLWK DQ
LQWHJUDWHG FRPSRQHQW IRU GHULYLQJ WKH
VHQVRU¶V DEVROXWH RULHQWDWLRQ 7KLV
VHQVRU ZDV FRPELQHG ZLWK D
PLFURFRQWUROOHU ZLWK LQWHJUDWHG :LL
PRGXOH 7KH VHQVRU¶V DEVROXWH
RULHQWDWLRQLQ XQLW TXDWHUQLRQIRUPDWZDV
WUDQVPLWWHG ZLUHOHVVOHVV DW  +HU] DQG
GLUHFWO\PDSSHGRQWKHRULHQWDWLRQRIWKH
LQWHUDFWLRQ MRLQW 7KLV H[SHULPHQW FRXOG
FRUUHVSRQGWRDQLPSURYLVDWLRQVHWWLQJRQ
VWDJHLQZKLFKDKXPDQGDQFHUSHUIRUPV
D GXHW ZLWK DQ DUWLILFLDO GDQFHU 7KH
LPSURYLVDWLRQ ZDV FRQGXFWHG ZLWK WZR
GLIIHUHQW LWHUDWLRQ FRQILJXUDWLRQV
&RQILJXUDWLRQ  FRGLQJ LWHUDWLRQV
LQWHUDFWLRQ LWHUDWLRQV  &RQILJXUDWLRQ 
FRGLQJ LWHUDWLRQV  DQG LQWHUDFWLRQ
LWHUDWLRQV  7KHVH WZR FRQILJXUDWLRQV
ZHUH FKRVHQ DV H[DPSOHV LQ ZKLFK
LQWHUDFWLRQ DQG DXWRHQFRGLQJ ZRXOG
ZHDNO\ FRQILJXUDWLRQ  RU VWURQJO\
FRQILJXUDWLRQDOWHUWKHRULJLQDOPRWLRQ
IURP WKH PRWLRQ FDSWXUH UHFRUGLQJ ,Q
ERWKFDVHVWKHQXPEHURILWHUDWLRQVZDV
FKRVHQ VR WKDW DXWRHQFRGLQJ DIIHFWV
%RVFK%12KWWSVZZZERVFK
VHQVRUWHFFRPSURGXFWVVPDUWVHQVRUVEQR
$UGXLQR0.5
KWWSVZZZDUGXLQRFFHQ*XLGH0.5
GXULQJ WKH ODVW LWHUDWLRQ DOO WKH VNHOHWRQ
MRLQWRULHQWDWLRQVLQFOXGLQJWKHLQWHUDFWLYH
MRLQW7KLVUHPRYHGDQRWKHUZLVHREYLRXV
PLUURULQJ HIIHFW EHWZHHQ WKH MRLQW
RULHQWDWLRQRI WKHKXPDQ DQGWKDW RI WKH
VNHOHWRQ 7KH UHVXOWV RI WKHVH WZR
H[SHULPHQWV DUH DYDLODEOH RQOLQH DV
YLGHR
5. Conclusion and Outlook
%DVHG RQ WKH UHVXOWV REWDLQHG VR IDU LW
VHHPV FOHDU WKDW LWHUDWLYH DXWRHQFRGLQJ
LQFRPELQDWLRQ ZLWKLQWHUDFWLYH FRQWURO RI
LQGLYLGXDOMRLQWV RIIHUV DQLQWHUHVWLQJDQG
IOH[LEOH IRUP IRU FRQWUROOLQJ DQ DUWLILFLDO
GDQFHU7KLVDSSURDFK ZRUNV DFFHSWDEO\
ZHOO IRU FDVHV LQ ZKLFK WKH QXPEHU RI
'2) RI WKH FRQWURO LQWHUIDFH LV
GUDPDWLFDOO\ ORZHU WKDQ WKRVH RI WKH
DUWLILFLDOGDQFHUDQGZKHUHWKHIRFXVOLHV
RQWKH FUHDWLRQ RIV\QWKHWLF PRWLRQV WKDW
SUHVHUYH VRPH RI WKH VW\OLVWLF SURSHUWLHV
SUHVHQW LQ WKH RULJLQDO PRWLRQ FDSWXUH
UHFRUGLQJV )XUWKHUPRUH WKH DSSURDFK
RIIHUV DQ LQWHUHVWLQJ FRPELQDWLRQ RI
FRQYHQWLRQDOPRWLRQSOD\EDFNJHQHUDWLYH
SULQFLSOHV DQG LQWXLWLYH LQWHUDFWLRQ $
VLPLODU OHYHO RI LQWXLWLYHQHVV PLJKW EH
GLIILFXOWWR DFKLHYH ZKHQZRUNLQJGLUHFWO\
ZLWKLQWKHODWHQWVSDFHRIDXWRHQFRGHUV
%XWGHVSLWHWKLVLQWXLWLYHQHVVWKHUHLVVWLOO
D OHDUQLQJ FXUYH LQYROYHG IRU JDLQLQJ DQ
XQGHUVWDQGLQJ IRU WKH LQWHUSOD\ EHWZHHQ
WKH SURSHUWLHV RI WKH RULJLQDO PRWLRQ
UHFRUGLQJWKHDXWRHQFRGLQJPHFKDQLVP
DQG WKH HIIHFW RI LQWHUIHULQJ ZLWK ERWK RI
WKHP E\ FRQWUROOLQJ FHUWDLQ MRLQW
RULHQWDWLRQV WKURXJK LQWHUDFWLRQ 7KH
,PSURYLVDWLRQ&RGLQJ,WHUDWLRQV,QWHUDFWLRQ
,WHUDWLRQV
KWWSVYLPHRFRPHHGH
,PSURYLVDWLRQ&RGLQJ,WHUDWLRQV,QWHUDFWLRQ
,WHUDWLRQV
KWWSVYLPHRFRPHGFDIF
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


H[SHULPHQWVFRQGXFWHGVRIDUVKHGVRPH
OLJKWRQWKLVLQWHUSOD\
$QLPSRUWDQWLQVLJKWJDLQHGFRQFHUQVWKH
EDODQFH EHWZHHQ RULJLQDO PRWLRQ
PDWHULDO LQWHUDFWLRQ DQG DXWRHQFRGLQJ
7KLVEDODQFHLVTXLFNO\VKLIWHGWRZDUGV D
GRPLQDQFHRI WKH HIIHFWRIDXWRHQFRGLQJ
DV VRRQ DV WKH QXPEHU RI FRGLQJ
LWHUDWLRQVLVFKRVHQ KLJKHU WKDQ ,IWKLV
LV WKH FDVH DXWRHQFRGLQJ EULQJV WR WKH
IRUHIURQW WKH PRVW IUHTXHQWO\ RFFXUULQJ
PDWHULDO LQ WKH RULJLQDO UHFRUGLQJ ZKLOH
UHPRYLQJ HYHU\WKLQJ HOVH ,W LV LPSRUWDQW
WR NHHS WKLV HIIHFW LQ PLQG ZKHQ
UHFRUGLQJ QHZ PRYHPHQWV DQG WU\LQJ WR
HQVXUH WKDW VRPH RI WKHVH PRYHPHQWV
ZLOO UHDSSHDU LQ WKH EHKDYLRXU RI WKH
DUWLILFLDO GDQFHU $QRWKHU LPSRUWDQW
LQVLJKW FRQFHUQV WKH GLIIHUHQFH EHWZHHQ
WKHRULJLQDORULHQWDWLRQ RI DVNHOHWRQMRLQW
DQG WKH LQWHUDFWLYHO\ JHQHUDWHG
RULHQWDWLRQ 7KH ODUJHU WKLV GLIIHUHQFH LV
WKHVWURQJHUGRHV DXWRHQFRGLQJ DOWHUWKH
RULJLQDOO\ UHFRUGHG PRWLRQV XS WR WKH
SRLQW ZKHUH WKH DXWRHQFRGHU JHQHUDWHV
V\QWKHWLF PRWLRQV ZKLFK DUH HQWLUHO\
GLVVLPLODUWR WKHFXUUHQWO\ SOD\HG VHFWLRQ
RI D PRWLRQ UHFRUGLQJ %\ SD\LQJ
DWWHQWLRQ WR WKH PRWLRQV FXUUHQWO\
H[HFXWHG E\ WKH DUWLILFLDO GDQFHU D
KXPDQ SHUIRUPHU FDQ H[SORLW WKLV
SULQFLSOH DQG HLWKHU PDNH WKH DUWLILFLDO
GDQFHUUHSURGXFHPRWLIVIURPWKHRULJLQDO
PRWLRQ UHFRUGLQJ RU JHQHUDWH QRYHO
PRWLRQV7KHVHQRYHOPRWLRQV FDQ HLWKHU
HPHUJH DV UHVXOW RI WKH DXWRHQFRGHU
FKDLQLQJ WRJHWKHU RULJLQDOO\ XQUHODWHG
PRWLRQV RU E\ GHFRGLQJ  YHFWRUV IURP
UHJLRQV RI ODWHQW VSDFH WKDW KDYH EHHQ
VSDUVHO\ SRSXODWHG GXULQJ WUDLQLQJ 7KH
ODWWHU HIIHFW FDQ EH HQIRUFHG E\ PDNLQJ
WKHLQWHUDFWLYHO\FRQWUROOHGMRLQWV DVVXPH
XQUHDOLVWLF RULHQWDWLRQV $ ODVW LPSRUWDQW
LQVLJKWFRQFHUQVWKHUHODWLRQVKLSEHWZHHQ
WKHYHORFLW\RI WKH UHFRUGHG PRWLRQVDQG
WKDW RI WKH LQWHUDFWLYHO\ FRQWUROOHG MRLQWV
7KHUHDUHWZRDVSHFWVWRWKLV)LUVWLIWKH
UHFRUGHG PRWLRQV H[KLELWV KLJK YHORFLW\
WKHQ LW LV GLIILFXOW WR PDLQWDLQ D VSHFLILF
OHYHO RI VLPLODULW\ EHWZHHQ WKH
LQWHUDFWLYHO\ FRQWUROOHG RULHQWDWLRQ RI D
MRLQW DQG LWV RULJLQDO RULHQWDWLRQ $V
FRQVHTXHQFHWKHPRWLRQVRIWKHDUWLILFLDO
GDQFHUEHFRPHXQSUHGLFWDEOH6HFRQGLI
RQO\ D IHZ MRLQWV H[KLELW KLJK YHORFLW\ LQ
WKH RULJLQDO UHFRUGLQJ WKHQ VORZLQJ WKHLU
PRWLRQGRZQFDXVHVWKHUHPDLQLQJMRLQWV
WREHFRPHIDVWHU,QFRPELQDWLRQZLWKWKH
RSSRVLWH HIIHFW LH PDNLQJ RULJLQDOO\
VORZ MRLQWV PRYH IDVWHU LW EHFRPHV
SRVVLEOH WR SDUWLDOO\ RU IXOO\ DOWHU WKH
SDFLQJRIWKHRULJLQDOPRWLRQ
7KH DXWKRUV SODQ WR FRQWLQXH WKLV
UHVHDUFKDORQJVHYHUDO GLUHFWLRQV $VDQ
LPPHGLDWHQH[WVWHSWKHFXUUHQWYHUVLRQ
RIWKH LPSOHPHQWDWLRQ ZLOOEHWHVWHG ZLWK
D SURIHVVLRQDO GDQFHU ZKR ZLOO GHYHORS
VPDOOSHUIRUPDQFH VNHWFKHVWKDW FDQEH
VKRZQLQDSXEOLF GHPRQVWUDWLRQ /HWWLQJ
D GDQFHU H[SHULPHQW ZLWK WKLV V\VWHP
DQG GHYHORS VPDOO FKRUHRJUDSKLHV IRU LW
ZLOO JHQHUDWH DGGLWLRQDO LQVLJKWV IRU WKH
DXWKRUV DQG WKHUHE\ KHOS ZLWK WKH
V\VWHP¶VIXWXUHGHYHORSPHQW $V D PRUH
ORQJ WHUP JRDO WKH DXWKRUV SODQ WR
FRPELQH WKH FXUUHQW V\VWHP ZLWK
DOJRULWKPLF PHWKRGV IRU PRWLRQ
V\QWKHVLV7KHFXUUHQWLPSOHPHQWDWLRQIRU
FRQWUROOLQJWKHRULHQWDWLRQRIMRLQWVPDNHV
LW YHU\ VLPSOH WR LQWHJUDWH RWKHU VRXUFHV
RIFRQWUROWKDQLQWHUDFWLYHLQSXW2QHILUVW
DWWHPSWLQWKLVGLUHFWLRQZLOOEHEDVHGRQ
D VRXQG V\QWKHVLV V\VWHP WKDW WKH
DXWKRUV SUHYLRXVO\ GHYHORSHG IRU D
SHUIRUPDQFHZLWKD YLROLQLVW >@ 7KLV
VRXQGV\QWKHVLVV\VWHPHPSOR\VPXOWLSOH
VLPXODWHG PDVVVSULQJ V\VWHPV WKDW DUH
FRXSOHG ZLWK HDFK RWKHU %\ PRGHOOLQJ
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


WKH VNHOHWRQ RI WKH DUWLILFLDO GDQFHU DV D
PDVVVSULQJV\VWHPWKHVNHOHWRQFDQEH
DGGHGWRWKHVRXQGV\QWKHVLVV\VWHPYLD
DQ DGGLWLRQDO FRXSOLQJ %\ GRLQJ VR WKH
VNHOHWRQFDQQRWRQO\EHFRQWUROOHGE\DQ
DGGLWLRQDO JHQHUDWLYH PHFKDQLVP EXW LW
FDQ DOVR EH UHQGHUHG DXGLEOH WKURXJK
VRQLILFDWLRQ$IWHU DOO WKHFRXSOLQJRI WKH
PDVVVSULQJ V\VWHPV ZRUNV LQ ERWK
GLUHFWLRQV DQG E\ DGGLQJ D PHFKDQLVP
WKDWJHQHUDWHVKLJKHURUGHUKDUPRQLFVWR
WKHRVFLOODWLRQVFDXVHGE\WKHVNHOHWRQD
IHHGEDFNPHFKDQLVP FDQEHHVWDEOLVKHG
LQ ZKLFK WKH V\QWKHWLF PRWLRQV RI WKH
VNHOHWRQ DQG WKH V\QWKHWLF VRXQGV
PXWXDOO\LQIOXHQFHHDFKRWKHU
7R FRQFOXGH WKH SURSRVHG DSSURDFK RI
FRPELQLQJ DQ DXWRHQFRGHU ZLWK
LQWHUDFWLRQIRU FUHDWLQJ V\QWKHWLFPRWLRQV
IRU DQ DUWLILFLDO GDQFHU KDV VKRZQ WR EH
VXIILFLHQWO\ SURPLVLQJ WR ZDUUDQW IXUWKHU
UHVHDUFK%XWVLQFHWKHFXUUHQWV\VWHPLV
OLNHO\ DOVR VXLWDEOH IRU FRPELQLQJ
DXWRHQFRGLQJ ZLWK DQ\ UHFRUGHG RU
DOJRULWKPLFDOO\JHQHUDWHGGDWDWKHVFRSH
RI WKLV DSSURDFK PLJKW H[SDQG EH\RQG
GLJLWDO SXSSHWU\ DQG GDQFH 7KHUHIRUH
WKLVUHVHDUFKFDQSRWHQWLDOO\FRQWULEXWHWR
D FRQYHUJHQFH RI WZR GLIIHUHQW SUDFWLFHV
LQ JHQHUDWLYH DUW WKRVH WKDW IDYRXU WKH
LQYHQWLRQ RI LGLRV\QFUDWLF UXOHEDVHG
DOJRULWKPV DQG WKRVH WKDW DSSO\ VWDWH RI
WKHDUWGHHSOHDUQLQJPHWKRGV
6. References
>@%LVLJ'DQLHO³*UDQXODU'DQFH´
1LQWK &RQIHUHQFH RQ &RPSXWDWLRQ
&RPPXQLFDWLRQ $HVWKHWLFV  ; L$'6
±
>@ 5DKXO 0  ³5HYLHZ RQ PRWLRQ
FDSWXUH WHFKQRORJ\´ *OREDO -RXUQDO RI
&RPSXWHU6FLHQFHDQG7HFKQRORJ\
>@ :KDWOH\ 6DUDK DQG +HWW\ %ODGHV
 ³'LJLWDO 'DQFH´ %ORRPVEXU\
&RPSDQLRQ WR 'DQFH 6WXGLHV /RQGRQ
%ORRPVEXU\ '2, KWWSVGRL
2UJ
>@ .DUUHPDQ /DXUD  ³+RZ GRHV
PRWLRQ FDSWXUH PHGLDWH GDQFH"´ ,Q
&RQWHPSRUDU\ &KRUHRJUDSK\ ±
5RXWOHGJH
>@&DOYHUW7RP³7KH(YROXWLRQRI
6RIWZDUH IRU 'DQFH´ NH\QRWH OHFWXUH
SUHVHQWHG DW WKH FRQIHUHQFH &RUSRUHDO
&RPSXWLQJ $ 3HUIRUPDWLYH $UFKDHRORJ\
RI 'LJLWDO *HVWXUH 8QLYHUVLW\ RI 6XUUH\
*XLOGIRUG
>@ &DPXUUL $QWRQLR *XDOWLHUR 9ROSH
6WHIDQR 3LDQD 0DXUL]LR 0DQFLQL
5DGRVODZ 1LHZLDGRPVNL 1LFROD )HUUDUL
DQG&RUUDGR&DQHSD³7KHGDQFHU
LQ WKH H\H WRZDUGV D PXOWLOD\HUHG
FRPSXWDWLRQDO IUDPHZRUN RI TXDOLWLHV LQ
PRYHPHQW´ 3URFHHGLQJV RI WKH UG
,QWHUQDWLRQDO 6\PSRVLXP RQ 0RYHPHQW
DQG&RPSXWLQJ±
>@ 6WDYUDNLV (IVWDWKLRV $QGUHDV
$ULVWLGRX 0DULD 6DYYD 6WHSKDQLD
/RL]LGRX +LPRQD DQG <LRUJRV
&KU\VDQWKRX  ³'LJLWL]DWLRQ RI
F\SULRWIRONGDQFHV´(XUR0HGLWHUUDQHDQ
&RQIHUHQFH6SULQJHU±
>@ +VLHK &KL0LQ DQG $QQLH /XFLDQL
 ³*HQHUDWLQJ GDQFH YHUEV DQG
DVVLVWLQJ FRPSXWHU FKRUHRJUDSK\´
3URFHHGLQJV RI WKH WK $QQXDO $&0
LQWHUQDWLRQDO &RQIHUHQFH RQ 0XOWLPHGLD
±
>@ /HDFK -DPHV DQG 6FRWW 'HODKXQWD
 ³'DQFH EHFRPLQJ NQRZOHGJH
GHVLJQLQJ D GLJLWDO ³ERG\´´ /HRQDUGR 
±
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


>@ )GLOL $ODRXL 6DUDK &\ULOOH +HQU\
DQG &KULVWLDQ -DFTXHPLQ 
³3K\VLFDO PRGHOOLQJ IRU LQWHUDFWLYH
LQVWDOODWLRQV DQG WKH SHUIRUPLQJ DUWV´
,QWHUQDWLRQDO-RXUQDORI3HUIRUPDQFH$UWV
DQG'LJLWDO0HGLD±
>@ %LVLJ 'DQLHO DQG 3DEOR 3DODFLR
 ³1HXUDO 1DUUDWLYHV 'DQFH ZLWK
9LUWXDO%RG\([WHQVLRQV´3URFHHGLQJVRI
WKH UG ,QWHUQDWLRQDO 6\PSRVLXP RQ
0RYHPHQWDQG&RPSXWLQJ±
>@ &DUOVRQ .ULVWLQ 7KHFOD 6FKLSKRUVW
DQG 3KLOLSSH 3DVTXLHU  ³6FXGGOH
*HQHUDWLQJ 0RYHPHQW &DWDO\VWV IRU
&RPSXWHU$LGHG &KRUHRJUDSK\´ ,&&&
±
>@ &DUOVRQ .ULVWLQ 3KLOLSSH 3DVTXLHU
+HUEHUW+7VDQJ-RUGRQ3KLOOLSV7KHFOD
6FKLSKRUVW DQG 7RP &DOYHUW 
³&RFKRUHR $ JHQHUDWLYH IHDWXUH LQ
LGDQFH)RUPVIRUFUHDWLQJQRYHONH\IUDPH
DQLPDWLRQ IRU FKRUHRJUDSK\´
3URFHHGLQJVRIWKH 6HYHQWK,QWHUQDWLRQDO
&RQIHUHQFHRQ&RPSXWDWLRQDO&UHDWLYLW\
>@ 9RQ /DEDQ 5XGROI  0RGHUQ
HGXFDWLRQDO GDQFH 3ULQFHWRQ %RRN
&RPSDQ\3XE
>@ 0DOHWLF 9HUD  %RG\ 6SDFH
([SUHVVLRQ 7KH 'HYHORSPHQW RI 5XGROI
/DEDQ¶V0RYHPHQWDQG'DQFH&RQFHSWV
0RXWRQGH*UX\WH
>@ /DSRLQWH )UDQoRLV-RVHSK DQG
0DUWLQH eSRTXH  ³7KH GDQFLQJ
JHQRPH SURMHFW JHQHUDWLRQ RI D
KXPDQFRPSXWHU FKRUHRJUDSK\ XVLQJ D
JHQHWLF DOJRULWKP´ 3URFHHGLQJV RI WKH
WK DQQXDO $&0 LQWHUQDWLRQDO
FRQIHUHQFHRQ0XOWLPHGLD±
>@ &UQNRYLF)ULLV /XND DQG /RXLVH
&UQNRYLF)ULLV  ³*HQHUDWLYH
FKRUHRJUDSK\XVLQJGHHSOHDUQLQJ´DU;LY
SUHSULQWDU;LY
>@ .DVSHUVHQ (VEHUQ 7RUJDUG 'DZLG
*yUQ\ &XPKXU (UNXW DQG *HRUJH
3DODPDV  ³*HQHUDWLYH
&KRUHRJUDSKLHV 7KH 3HUIRUPDQFH
'UDPDWXUJ\ RI WKH 0DFKLQH´
3URFHHGLQJV RI WKH WK ,QWHUQDWLRQDO
-RLQW &RQIHUHQFH RQ &RPSXWHU 9LVLRQ
,PDJLQJDQG &RPSXWHU*UDSKLFV 7KHRU\
DQG $SSOLFDWLRQV9ROXPH  *5$33
6&,7(35(66'LJLWDO/LEUDU\±
>@ 3HWWHH 0DULHO &KDVH 6KLPPLQ
'RXJODV'XKDLPHDQG,O\D9LGULQ
³%H\RQG LPLWDWLRQ *HQHUDWLYH DQG
YDULDWLRQDO FKRUHRJUDSK\ YLD PDFKLQH
OHDUQLQJ´ DU;LY SUHSULQW
DU;LY
>@ %HUPDQ $OH[DQGHU DQG 9DOHQFLD
-DPHV  ³7RZDUGV D OLYH GDQFH
LPSURYLVDWLRQ EHWZHHQ DQ DYDWDU DQG
DKXPDQ GDQFHU´ 3URFHHGLQJV RI WKH
 ,QWHUQDWLRQDO :RUNVKRS RQ
0RYHPHQWDQG&RPSXWLQJ±
>@ %HUPDQ $OH[DQGHU DQG 9DOHQFLD
-DPHV  ³.LQHWLF 'LDORJXHV
(QKDQFLQJ FUHDWLYLW\ LQ GDQFH´
3URFHHGLQJV RI WKH QG ,QWHUQDWLRQDO
:RUNVKRSRQ0RYHPHQWDQG&RPSXWLQJ
±
>@ %HUPDQ $OH[DQGHU DQG 9DOHQFLD
-DPHV³/HDUQLQJDV3HUIRUPDQFH
$XWRHQFRGLQJ DQG *HQHUDWLQJ 'DQFH
0RYHPHQWV LQ 5HDO 7LPH´ ,QWHUQDWLRQDO
&RQIHUHQFH RQ &RPSXWDWLRQDO
,QWHOOLJHQFH LQ 0XVLF 6RXQG $UW DQG
'HVLJQ6SULQJHU±
>@0F&RUPLFN -RKQ .LP9LQFV6DHLG
1DKDYDQGL DQG 'RXJODV &UHLJKWRQ
³/HDUQLQJWRGDQFHZLWKDKXPDQ´
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH


>@0F&RUPLFN -RKQ .LP9LQFV6DHLG
1DKDYDQGL 'RXJODV &UHLJKWRQ DQG
6WHSK +XWFKLVRQ  ³7HDFKLQJ D
GLJLWDO SHUIRUPLQJ DJHQW $UWLILFLDO QHXUDO
QHWZRUN DQG KLGGHQ PDUNRY PRGHO IRU
UHFRJQLVLQJ DQG SHUIRUPLQJ GDQFH
PRYHPHQW´ 3URFHHGLQJV RI WKH 
,QWHUQDWLRQDO :RUNVKRS RQ 0RYHPHQW
DQG&RPSXWLQJ±
>@ 0F&RUPLFN -RKQ 6WHSK
+XWFKLQVRQ.LP9LQFVDQG-RUGDQ%HWK
9LQFHQW  ³(PHUJHQW EHKDYLRXU
OHDUQLQJ IURP DQ DUWLILFLDOO\ LQWHOOLJHQW
SHUIRUPLQJ VRIWZDUH DJHQW´ ,6($ 
3URFHHGLQJV RI WKH VW ,QWHUQDWLRQDO
6\PSRVLXP RQ (OHFWURQLF $UW ,6($
±
>@ /LX /XFDV 'XUL /RQJ 6ZDU
*XMUDQLD DQG %ULDQ 0DJHUNR 
³/HDUQLQJ PRYHPHQW WKURXJK KXPDQ
FRPSXWHU FRFUHDWLYH LPSURYLVDWLRQ´
3URFHHGLQJV RI WKH WK ,QWHUQDWLRQDO
&RQIHUHQFH RQ 0RYHPHQW DQG
&RPSXWLQJ±
>@ -DFRE 0LNKDLO DQG %ULDQ 0DJHUNR
 ³,QWHUDFWLRQEDVHG $XWKRULQJ IRU
6FDODEOH &RFUHDWLYH $JHQWV´ ,&&&
±
>@ +DELELH ,NKVDQXO 'DQLHO +ROGHQ
-RQDWKDQ 6FKZDU] -RH <HDUVOH\ DQG
7DNX .RPXUD  ³$ UHFXUUHQW
YDULDWLRQDO DXWRHQFRGHU IRU KXPDQ
PRWLRQ V\QWKHVLV´ WK %ULWLVK 0DFKLQH
9LVLRQ&RQIHUHQFH
>@ +ROGHQ 'DQLHO -XQ 6DLWR 7DNX
.RPXUD DQG 7KRPDV -R\FH 
³/HDUQLQJ PRWLRQ PDQLIROGV ZLWK
FRQYROXWLRQDO DXWRHQFRGHUV´ ,Q
6,**5$3+ $VLD  7HFKQLFDO %ULHIV
±
>@+ROGHQ 'DQLHO-XQ6DLWRDQG7DNX
.RPXUD  ³$ GHHS OHDUQLQJ
IUDPHZRUNIRUFKDUDFWHUPRWLRQV\QWKHVLV
DQG HGLWLQJ´ $&0 7UDQVDFWLRQV RQ
*UDSKLFV72*±
>@ :DQJ 4L 7KLHUU\ $UWLqUHV 0LFNDHO
&KHQ DQG /XGRYLF 'HQR\HU 
³$GYHUVDULDOOHDUQLQJIRUPRGHOLQJKXPDQ
PRWLRQ´ 7KH 9LVXDO &RPSXWHU  
±
>@ /L =LPR <L =KRX 6KXDQJMLX ;LDR
&KRQJ +H =HQJ +XDQJ DQG +DR /L
 ³$XWRFRQGLWLRQHG UHFXUUHQW
QHWZRUNV IRU H[WHQGHG FRPSOH[ KXPDQ
PRWLRQ V\QWKHVLV´ DU;LY SUHSULQW
DU;LY
>@ GH *UDI %UDG DQG (PUH <LOPD]
 ³3XSSHWRORJ\ 6FLHQFH RU FXOW´
$QLPDWLRQ:RUOGQR
>@GD&RVWD/HLWH/XtV0LJXHO%DUERVD
 ³YLUWXDO PDULRQHWWH LQWHUDFWLRQ
PRGHOIRUGLJLWDOSXSSHWU\´
>@ 2VKLWD 0DVDNL <XWD 6HQMX DQG
6\XQ0RULVKLJH³&KDUDFWHUPRWLRQ
FRQWURO LQWHUIDFH ZLWK KDQG PDQLSXODWLRQ
LQVSLUHG E\ SXSSHW PHFKDQLVP´
3URFHHGLQJV RI WKH WK $&0
6,**5$3+ ,QWHUQDWLRQDO &RQIHUHQFH RQ
9LUWXDO5HDOLW\ &RQWLQXXP DQG ,WV
$SSOLFDWLRQVLQ,QGXVWU\±
>@ +D 6HKRRQ <XQIHL %DL DQG &
.DUHQ /LX  ³+XPDQ PRWLRQ
UHFRQVWUXFWLRQ IURP IRUFH VHQVRUV´
3URFHHGLQJV RI WKH  $&0
6,**5$3+(XURJUDSKLFV6\PSRVLXPRQ
&RPSXWHU$QLPDWLRQ±
>@ ,VKLJDNL 6DWRUX 7LPRWK\ :KLWH
9LFWRU%=RUGDQDQG & .DUHQ/LX
³3HUIRUPDQFHEDVHGFRQWURO LQWHUIDFHIRU
FKDUDFWHU DQLPDWLRQ´ $&0 7UDQVDFWLRQV
RQ*UDSKLFV72*±
>@ 2X]RXQLV &KULVWRV $ &KULVWRV
0RXVDV &KULVWRV1LNRODRV
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH

$QDJQRVWRSRXORV DQG 3DXO 1HZEXU\
 ³8VLQJ 3HUVRQDOL]HG )LQJHU
*HVWXUHV IRU 1DYLJDWLQJ 9LUWXDO
&KDUDFWHUV´95,3+<6±
>@:DQJ<XPHQJ:XMXQ&KHDQG%R
;X  ³(QFRGHU±GHFRGHU UHFXUUHQW
QHWZRUN PRGHO IRU LQWHUDFWLYH FKDUDFWHU
DQLPDWLRQ JHQHUDWLRQ´ 7KH 9LVXDO
&RPSXWHU±
>@ 5RDGV &XUWLV  0LFURVRXQG
0,7SUHVV
>@6FKZDU]'LHPRHWDO³'DWD
GULYHQFRQFDWHQDWLYHVRXQGV\QWKHVLV´
>@ =LOV $\PHULF DQG )UDQoRLV 3DFKHW
 ³0XVLFDO PRVDLFLQJ´ 'LJLWDO $XGLR
(IIHFWV'$)[9ROXPH&LWHVHHU
>@ 6KRHPDNH .HQ  ³$QLPDWLQJ
URWDWLRQ ZLWK TXDWHUQLRQ FXUYHV´
3URFHHGLQJV RI WKH WK DQQXDO
FRQIHUHQFH RQ &RPSXWHU JUDSKLFV DQG
LQWHUDFWLYHWHFKQLTXHV±
>@%LVLJ 'DQLHO DQG(SKUDLP:HJQHU
 ³6WULQJV´ (LJKWK &RQIHUHQFH RQ
&RPSXWDWLRQ&RPPXQLFDWLRQ$HVWKHWLFV
;L$'6±
>@%LVLJ 'DQLHO (SKUDLP:HJQHUDQG
+DUDOG .LPPLJ  ³6WULQJV 3´ 1LQWK
&RQIHUHQFH RQ &RPSXWDWLRQ
&RPPXQLFDWLRQ $HVWKHWLFV  ; L$'6
±
X
XIV Generative Art Conference - GA2021
;
;,9*HQHUDWLYH$UW&RQIHUHQFH*$
S
DJH
