Song, Miao (2012) Computer-Assisted Interactive Documentary and Performance Arts in Illimitable Space. PhD thesis, Concordia University.
Preview |
Text (Final version) (application/pdf)
24MBSong_PhD_S2013.pdf - Accepted Version Available under License Creative Commons Attribution Non-commercial Share Alike. |
Abstract
This major component of the research described in this thesis is 3D computer graphics, specifically the realistic physics-based softbody simulation and haptic responsive environments. Minor components include advanced human-computer interaction environments, non-linear documentary storytelling, and theatre performance. The journey of this research has been unusual because it requires a researcher with solid knowledge and background in multiple disciplines; who also has to be creative and sensitive in order to combine the possible areas into a new research direction. Thus, we summarize the innovative research work surrounding the topic as “Computer-Assisted Interactive Documentary and Performance Arts in Illimitable Space”. This work encompasses a lot of research performed in each of the disciplines. It focuses on the advanced computer graphics and emerges from experimental cinematic works and theatrical artistic practices. Some development content and installations are completed to prove and evaluate the described concepts and to be convincing.
More specifically, on one hand, the major research component includes the advanced rendering in real-time of an interactive physics-based softbody object simulation and visualization with OpenGL. Its immediate follow-up work in this thesis extends that system onto an artistic interactive jellyfish simulation controlled with advanced interaction devices, such as Falcon haptics. Some more advanced rendering techniques have been applied, such as stereoscopic effects, GLSL, LOD etc. in order to bring the CG objects to life and to increase the level of realism and speed.
On the other hand, the installation work, /Tangible Memories/ transforms the award-winning personal documentary film /I Still Remember/ [1], into a non-linear interactive audience-controlled piece using the new media technologies and devices. The audience and society have already appreciated the personal documentary /I Still Remember/ with the social values resulting in portrayal of immigration, divorce, reunification with the family, and other memories. Turning it into an interactive new media work makes it a much more profound and sensory in-depth storytelling approach that can be educational as well as help the audience to feel the story by interacting with it and making it available via many media sources. Moreover, the audience's participations and feedback are themselves well-preserved in the new "memory bubbles", so that the same documentary project could be eternal and ever evolving, which may determine the concept of tomorrow's documentary film production. Additionally, another audacious approach extended from /Tangible Memories/ installation is for theatrical practice with the same set of technical tools. Theater performers could use their body movements, gestures, and facial expressions to achieve the perceptual and emotional digital effects in sound and images dynamically.
To summarize, the resulting work involves not only artistic creativity, but solving or combining technological hurdles in motion tracking, pattern recognition, force feedback control, etc., with the available documentary footage on film, video, or images, and text via a variety of devices (input and output, projection, stereoscopic viewing) and programming, and installing all the needed interfaces such that it all works in real-time. Thus, the contribution to the knowledge advancement is in solving these interfacing problems and the real-time aspects of the interaction that have uses in film industry, fashion industry, new age interactive theatre, computer games, and web-based technologies and services for entertainment and education. It also includes building up on this experience to integrate Kinect- and haptic-based interaction, artistic scenery rendering, and other forms of control. This research work connects all the research disciplines, seemingly disjoint fields of research, such as computer graphics, documentary film, interactive media, and theatre performance together.
| Divisions: | Concordia University > Gina Cody School of Engineering and Computer Science > Computer Science and Software Engineering Concordia University > Faculty of Fine Arts > Mel Hoppenheim School of Cinema Concordia University > Faculty of Fine Arts > Theatre Concordia University > Research Units > Hexagram - The Institute for Research/Creation in Media Arts and Technologies Concordia University > School of Graduate Studies |
|---|---|
| Item Type: | Thesis (PhD) |
| Authors: | Song, Miao |
| Institution: | Concordia University |
| Degree Name: | Ph. D. |
| Program: | Special Individualized Program |
| Date: | 24 December 2012 |
| Thesis Supervisor(s): | Grogono, Peter |
| Keywords: | interactive documentary, illimitable space, interactive performance, HCI in theatre, softbody simulation, OpenGL, XNA, Kinect, Falcon, haptics, jellyfish simulation, real-time, computer graphics, modeling and simulation |
| ID Code: | 975072 |
| Deposited By: | MIAO SONG |
| Deposited On: | 18 Jun 2013 11:56 |
| Last Modified: | 18 Jan 2018 17:39 |
| Related URLs: |
References:
[1] Miao Song (Director). I Still Remember. 1st BJIFF “See the world through films” Contest for Best Documentary Short, 2011. [Documentary film]; 13 minutes; Best Documentary Award. http://www.bjiff.com/en/bjiffnews/n214618230.shtml.[2] Alison Reiko Loader. Making space. Master’s thesis, Department of Design and Computation Arts, Concordia University, Montreal Canada, 2008.
[3] Novint Technologies, Inc. Novint Falcon. [online], 2011. http://www.novint.com/index.php/products/novintfalcon.
[4] B. C. Choi and A. W. Pak. Multidisciplinarity, interdisciplinarity and transdisciplinarity in health research, services, education and policy. [online], 2006. http://www.ncbi.nlm.nih.gov/pubmed/17330451, last viewed June 2010.
[5] Miao Song. Dynamic deformation of uniform elastic two-layer objects. Master's thesis, Department of Computer Science and Software Engineering, Concordia University, Montreal, Canada, August 2007. ISBN: 978-0-4943-4780-5, http://arxiv.org/abs/0907.4364.
[6] Jason Lewis and OBX Labs. CitySpeak. http://cspeak.net/, 2008–2012.
[7] Miao Song, Serguei A. Mokhov, Alison R. Loader, and Maureen J. Simmonds. Astereoscopic OpenGL-based interactive plug-in framework for Maya and beyond. In Proceedings of VRCAI’09, pages 363–368, New York, NY, USA, 2009. ACM.
[8] Alison R. Loader, Serguei A. Mokhov, and Miao Song. Open Stereoscopic 3D Plugin Collection. SourceForge.net, 2008–2012. http://sf.net/projects/stereo3d, last viewed November 2010.
[9] Autodesk. Maya. [digital], 2008–2012. autodesk.com.
[10] Song Wang, Miao Song, Zhang Ling, and Maureen J. Simmonds. Pain and performance in virtual reality environments: A pilot feasibility study. Poster at the 3rd Pain, Mind and Movement Symposium, August 2010.
[11] Miao Song and Peter Grogono. Real-time modeling and physical-based animation of a jellyfish from softbody in OpenGL. In Bipin C. Desai, Sudhir P. Mudur, and Emil I. Vassev, editors, Proceedings of the Fifth International C* Conference on Computer Science and Software Engineering (C3S2E’12), pages 123–124, New York, NY, USA, June 2012. ACM. Poster.
[12] Miao Song, Maureen J. Simmonds, and Peter Grogono. Innovative medical applications and beyond of 3D techniques in a responsive virtual reality lab: Experience report. Unpublished, December 2010.
[13] Miao Song. Interactive Elastic Two-Layer Soft Body Simulation with OpenGL. Lambert Academic Publishing, June 2010. ISBN: 978-3-8383-4137-8.
[14] Miao Song and Peter Grogono. A framework for dynamic deformation of uniform elastic two-layer 2D and 3D objects in OpenGL. In Proceedings of C3S2E’08, pages 145–158. ACM, May 2008.
[15] Miao Song, Peter Grogono, Jason Lewis, and Maureen J. Simmonds. A poor woman’s interactive remake of the “I Still Remember” documentary with OpenGL. In Junia Anacleto, Sidney Fels, Nicholas Graham, Bill Kapralos, Magy Seif El-Nasr, and Kevin Stanley, editors, Proceedings of ICEC 2011, number 6972 in LNCS, pages 362–366. Springer, October 2011.
[16] Miao Song (Director). I Still Remember. Showcased in HTMlles 2010, 2010. [Documentary film]; 9 minutes; http://www.htmlles.net/2010/projects/emergence/.
[17] Miao Song. Interdisciplinary research presentation related to research-creation media arts. Concordia University Hexagram, December 2010.
[18] Miao Song and Peter Rist. Water ink animation film – a career of animation, complex for water-ink. Concordia University, 2011. An Essay Translation from book: “Between Looking up and Stooping: the Memory of Three Generation Female Chinese Film Photographers”.
[19] Miao Song and Peter Grogono. Are haptics-enabled interactive and tangible cinema, documentaries, 3D games, and specialist training applications our future? In Proceedings of GRAPP’09, pages 393–398. INSTICC, February 2009.
[20] Miao Song. The role of computer graphics in documentary film. Presentation at FSAC (Film Studies Association of Canada) 2009, Carleton University, Ottawa, May 2009. http://www.filmstudies.ca/FSAC_conference2009.pdf.
[21] Miao Song. Unraveling Her Story: Miao. Theatre Production, 2007. Directed by Emily Burkes-Nossiter, Co-directed by Chia-Wen, Scriptwriting and acting by Lois Jones, Lucy Lu, Joy Ruben, Dorothy Singer, Miao Song, Talia Weisz, and Mimi Zhou.
[22] Miao Song and Peter Grogono. Application of advanced rendering and animation techniques for 3D games to softbody modeling and animation. In Proceedings of C3S2E’09, pages 89–100, Montreal, Quebec, Canada, May 2009. ACM.
[23] Miao Song and Peter Grogono. Deriving software engineering requirements specification for computer graphics simulation systems through a case study. In Proceedings of the 3rd International Conference on Information Sciences and Interaction Sciences (ICIS2010), pages 285–291. IEEE Computer Society, June 2010.
[24] Wikipedia. Web documentary — Wikipedia, The Free Encyclopedia, 2012. [Online; accessed 3-August-2012].
[25] Marian F. Ursu, Vilmos Zsombori, John Wyver, Lucie Conrad, Ian Kegel, and Doug Williams. Interactive documentaries: A golden age. Comput. Entertain., 7:41:1–41:29, September 2009.
[26] Florian Thalhofer. The Korsakow system: Database-driven interactive Korsakow films production. [online], 2008–2011. http://korsakow.org.
[27] Katerina Cizek (Director). HIGHRISE: Out of my window. [online], National Film Board of Canada, 2008–2010. A 360-degree interactive documentary, http://highrise.nfb.ca.
[28] Jimmy Wales, Larry Sanger, and other authors from all over the world. Wikipedia: The free encyclopedia. [online], Wikimedia Foundation, Inc., 2001–2012. http://wikipedia.org.
[29] Wayne Carlson. A critical history of computer graphics and animation. [online; accessed 2012], 2003. http://design.osu.edu/carlson/history/lessons.html.
[30] John A. Lent, editor. Animation in Asia and the Pacific. Indiana University Press, July 2001. ISBN: 978-025-334-035-1.
[31] Donald Hearn, M. Pauline Baker, and Warren Carithers. Computer Graphics with OpenGL. Prentice Hall, 4 edition, November 2010. ISBN: 978-0136053583.
[32] Tomas Akenine-M¨oller and Eric Haines. Real-Time Rendering. A.K. Peters Ltd., 2 edition, 2002. ISBN 1568811829, http://realtimerendering.com/index_rtr2.html.
[33] Natalya Tatarchuk, editor. Advanced real-time rendering in 3D graphics and games, 2006. http://developer.amd.com/media/gpu_assets/Course_26_SIGGRAPH_2006.pdf.
[34] Tan Tiow Seng. CS5243 3D game programming technology class notes. School of Computing, National University of Singapore, 2007. http://www.comp.nus.edu.sg/~cs5243/.
[35] Sudhir P. Mudur. COMP7661 advanced rendering and animation class notes. Concordia University, September 2007.
[36] Randi J. Rost. OpenGL Shading Language. Pearson Education, Inc., February 2004. ISBN: 0-321-19789-5.
[37] Frank Losasso. Surface reflection models. NVIDIA Corporation, 2004.
[38] Wikipedia. Toy Story — Wikipedia, The Free Encyclopedia. [Online; accessed 15- August-2008], http://en.wikipedia.org/w/index.php?title=Toy_Story&oldid=232017683, 2008. [motion picture].
[39] Paul Lincoln. Citizens comfort. [online]. http://www.mediartchina.org/recomb/citizen.
[40] Xu Zhongmin. Cloud. [online]. http://www.dgd.stu.edu.tw/DGD3/index-1-tst001.html.
[41] Fu ZhiYong. A swimming fish. [online]. http://newmediabeijing.org/md2004/exhibition_display.php?section=2&link_id=1&title=The+Swimming+Fish&artist_id=.
[42] E-GO Computer Graphics. Tsinghua university 100 anniversary celebration building projection, May 2011. http://www.e-go-cg.com/index.html.
[43] SIGGRAPH. SIGGRAPH ASIA ’10: ACM SIGGRAPH Asia 2010 papers, NewYork, NY, USA, 2010. ACM.
[44] SIGGRAPH. SA ’10: ACM SIGGRAPH ASIA 2010 Computer Animation Festival, New York, NY, USA, 2010. ACM.
[45] SIGGRAPH. SA ’10: ACM SIGGRAPH ASIA 2010 Art + Tech Showcase, New York, NY, USA, 2010. ACM.
[46] SIGGRAPH. SIGGRAPH ASIA ’09: ACM SIGGRAPH ASIA 2009 Art Gallery & Emerging Technologies: Adaptation, New York, NY, USA, 2009. ACM.
[47] SIGGRAPH. SIGGRAPH ASIA ’09: ACM SIGGRAPH ASIA 2009 Computer Animation Festival, New York, NY, USA, 2009. ACM.
[48] SIGGRAPH. SIGGRAPH Asia ’08: ACM SIGGRAPH ASIA 2008 computer animation festival, New York, NY, USA, 2008. ACM.
[49] SIGGRAPH. SIGGRAPH Asia ’08: ACM SIGGRAPH ASIA 2008 artgallery: emerging technologies, New York, NY, USA, 2008. ACM.
[50] John C. Hart, editor. SIGGRAPH Asia ’08: ACM SIGGRAPH Asia 2008 papers, New York, NY, USA, 2008. SIGGRAPH, ACM.
[51] Isaac Victor Kerlow. The Art of 3-D Computer Animation and Imaging. John Wiley & Sons, Inc., New York, NY, USA, 2 edition, 2000. ISBN: 0-471-36004-X.
[52] Lee Montgomery. Tradigital Maya: A CG Animator’s Guide to Applying the Classical Principles of Animation. Focal Press, 1 edition, 2011. ISBN 0123852226, 9780123852229.
[53] Wikipedia. Procedural modeling — Wikipedia, The Free Encyclopedia. [online; accessed 27-August-2011], 2011. http://en.wikipedia.org/w/index.php?title=Procedural_modeling&oldid=432153189.
[54] Manfred Lau and James J. Kuffner. Behavior planning for character animation. Eurographics ACM SIGGRAPH Symposium on Computer Animation, 2005.
[55] William T. Reeves. Particle systems – a technique for modeling a class of fuzzy objects. Computer Graphics, 17:359–376, 1983.
[56] Paul Bourke. Particle System Example. The University of Wester Australia, 1998. http://local.wasp.uwa.edu.au/~pbourke/modelling_rendering/particle/index.html.
[57] Andrew Witkin. Particle system dynamics. In SIGGRAPH97 Course Notes. ACM, 1997. Online at http://www.cs.cmu.edu/afs/cs/user/baraff/www/sigcourse/notesc.pdf.
[58] Jernej Barbiˇc, Marco da Silva, and Jovan Popovi´c. Deformable object animation using reduced optimal control. ACM Trans. Graph., 28:1–9, July 2009.
[59] Jernej Barbiˇc. Vega – 3D deformable object library. [online], August 2012. http://run.usc.edu/vega/, viewed August 2012.
[60] Hugh Hancock. Machinima cutscene creation, part one. Gamasutra, September 2000. http://www.gamasutra.com/view/feature/131537/machinima_cutscene_creation_part_.php.
[61] Phylis Johnson and Donald Pettit. Machinima: The Art and Practice of Virtual Filmmaking. McFarland Press, 2011.
[62] Jenna Ng, editor. Understanding Machinima: Essays in Film-making in Virtual Worlds. Continuum Press, New York, NY, 2012.
[63] Wikipedia. Quake (video game) — wikipedia, the free encyclopedia. [Online; accessed 14-September-2012], 2012.
[64] Eike Falk Anderson. Off-line evolution of behaviour for autonomous agents in realtime computer games. In H.-P. Schwefel, J.-J. Merelo Guerv´os, P. Adamidis, H.-G. Beyer, and J.-L. Fern´andez-Villaca˜nas, editors, Parallel Problem Solving from Nature (PPSN VII), number 2439 in Lecture Notes in Computer Science (LNCS), page 689. Springer-Verlag, 2002. http://link.springer.de/link/service/series/0558/papers/2439/243900689.pdf.
[65] Michael Nikonov and iπ Soft. iPi Desktop Motion Capture. [online], 2008–2012. http://ipisoft.com.
[66] Wikipedia. Soft body dynamics — Wikipedia, The Free Encyclopedia. [online; accessed 10-September-2012], 2012. http://en.wikipedia.org/wiki/Soft_body_dynamics.
[67] M. Moore and J. Wilhelms. Collision detection and response for computer animation. Computer Graphics, 22(4):289–298, August 1988.
[68] Ming Chieh Lin. Efficient Collision Detection for Animation and Robotics. PhD thesis, Department of Electrical Engineering and Computer Science, University of California at Berkeley, 1993. Online at: ftp://ftp.cs.unc.edu/pub/users/manocha/PAPERS/COLLISION/thesis.ps.Z.
[69] Chris Hecker. Physics, part 3: Collision response. Game Developer Magazine, pages 11–18, March 1997.
[70] Matthias Muller, Julie Dorsey, Leonard McMillan, and Robert Jagnow. Collisions and deformations: Stable real-time deformations. In Proceedings of the 2002 ACMSIGGRAPH/Eurographics symposium on Computer Animation, pages 49–54. ACM, July 2002.
[71] Steve Capell, Seth Green, Brain Curless, and Zoran Popvic. Collisions and deformations: A multi resolution framework for dynamic deformations. In Proceedings of
the 2002 ACM SIGGRAPH/Eurographics symposium on Computer Animation, pages 41–48. ACM, July 2002.
[72] Wikipedia. Rendering (Computer Graphics). Wikipedia, the free encyclopedia, 2012. http://en.wikipedia.org/wiki/Rendering_(computer_graphics).
[73] Alan Watt and Fabio Policarpo. 3D Games: Real-time Rendering and Software Technology, volume I of ACM Press SIGGRAPH series. Addison-Wesley, 1 edition, 2001.
Edited by Stephen Spencer.
[74] Alan Watt and Fabio Policarpo. 3D Games Animation and Advanced Real-time Rendering. Addison-Wesley, 2 edition, 2003. ISBN 0-201-78706-7.
[75] Jonathan Gibbs. Rendering Skin and Hair. ACM, March 2001. http://silicon-valley.siggraph.org/MeetingNotes/shrek/hairskin.pdf.
[76] Barbara Robinson. The first CG cast of human actors stars in Final Fantasy: The Spirits Within. Computer Graphics World, August 2001. http://cgw.pennnet.com/Articles/Article_Display.cfm?Section=Archives&\
Subsection=Display&ARTICLE_ID=108473.
[77] Kayvon Fatahalian and Tim Foley. Rendering jellyfish. [online], June 2004. http://graphics.stanford.edu/courses/cs348b-competition/cs348b-04/jellies/index.html, viewed January 2011.
[78] Patrick Coleman and Karan Singh. Ryan: rendering your animation nonlinearly projected. In NPAR ’04: Proceedings of the 3rd international symposium on Nonphotorealistic
animation and rendering, pages 129–156, New York, NY, USA, 2004. ACM.
[79] Janne Kontkanen. Novel Illumination Algorithms For Off-Line And Real-Time Rendering. PhD thesis, Helsinki University of Technology, 2007. http://lib.tkk.fi/
Diss/2007/isbn9789512286102/isbn9789512286102.pdf.
[80] Tomas Moller and Eric Haines. Real-Time Rendering. A. K. Peters Limited, 2 edition, 2002. ISBN 1568811829.
[81] Michael Repplinger, Alexander L¨offler, Dmitri Rubinstein, and Philipp Slusallek. URay: A flexible framework for distributed rendering and display. Technical
Report TR-2008-01, Computer Graphics Group, Saarland University, December 2008. http://graphics.cg.uni-saarland.de/fileadmin/cguds/papers/2008/repplinger_2008/tr_2008-1_uray.pdfs.
[82] Wikipedia. Global illumination. Wikipedia, the free encyclopedia, 2008. http://en.wikipedia.org/wiki/Global_illumination.
[83] Wikipedia. Level of detail — Wikipedia, The Free Encyclopedia. [Online; accessed 29-October-2012], 2012. http://en.wikipedia.org/w/index.php?title=Level_of_
detail&oldid=495068316.
[84] Carl M. Erikson. Hierarchical Levels of Detail to Accelerate the Rendering of Large Static and Dynamic Polygonal Environments. PhD thesis, University of North Carolina at Chapel Hill, 2000. https://wwwx.cs.unc.edu/~geom/papers/documents/dissertations/erikson00.pdf.
[85] Maryann Simmons and Dave Shreiner. Per-pixel smooth shader level of detail. [online], 2003. http://i31www.ira.uka.de/~semin05/LevelOfDetail/Material/p1-simmons.pdf.
[86] Daniel R´akos. GPU based dynamic geometry LOD. [online], October 2010. http://rastergrid.com/blog/2010/10/gpu-based-dynamic-geometry-lod/.
[87] Willem H. de Boer. Fast terrain rendering using geometrical mipmapping. [online], October 2000. http://www.flipcode.com/archives/article_geomipmaps.pdf.
[88] Kurt Akeley, Allen Akin, Ben Ashbaugh, Bob Beretta, John Carmack, Matt Craighead, Ken Dyke, Steve Glanville, Michael Gold, Evan Hart, Mark Kilgard, Bill Licea-Kane, Barthold Lichtenbelt, Erik Lindholm, Benj Lipchak, Bill Mark, James McCombe, Jeremy Morris, Brian Paul, Bimal Poddar, Thomas Roell, Jeremy Sandmel, Jon Paul Schelter, Geoff Stahl, John Stauffer, and Nick Triantos. ARB vertex program, Revision 46. [online], 2002–2007. NVIDIA Corporation, ARB Extension #26, http://www.opengl.org/registry/specs/ARB/vertex_program.txt.
[89] Bob Beretta, Pat Brown, Matt Craighead, Cass Everitt, Evan Hart, Jon Leech, Bill Licea-Kane, Bimal Poddar, Jeremy Sandmel, Jon Paul Schelter, Avinash Seetharamaiah, Nick Triantos, and contributors to the ARB vertex program working
group. ARB fragment program, Revision 27. [online], 2002–2006. Microsoft Corporation, ARB Extension #27, http://www.opengl.org/registry/specs/ARB/fragment_program.txt.
[90] OpenGL. GLSL Quick Reference Guide. [online], 2008. http://www.opengl.org/sdk/libsOpenSceneGraphglsl_quickref.pdf.
...
All items in Spectrum are protected by copyright, with all rights reserved. The use of items is governed by Spectrum's terms of access.
Repository Staff Only: item control page
Download Statistics
Download Statistics