2) The computing layer:

· Processing

· Sorting and matching

· Function and variable

· Computer language

· Data structures (database)

New Media Cultural Forms

1) Algorithmic – The virtual computer-based space (2-D GUI, 3-D navigable environments such as video games)

2) The Database (search, match, sort, filter, are the standard methods of navigation)

The database is a standard form of contemporary culture. Data now includes images, audio, video, text, statistics, etc., stored in structures such as arrays (one dimension = vector, two dimensions = matrix), linked lists, tables, and graphs.

Navigation through data structures is the task for researchers using new media. Database navigation schemes can be considered as informational or knowledge narratives.

Some examples of web based database narratives are:

They Rule (http://www.theyrule.net/) flash site linked to Fortune 100 database and other web resources.
Plumbdesign 3-D English Thesaurus (http://www.visualthesaurus.com/online/index.html)
Total information awareness program (http://www.darpa.mil/iao/TIASystems.htm)
Search engines, web crawlers Searching 3,083,324,652 web pages (http://www.google.ca/)
Internet archive (http://archive.org/)
Library of Congress (http://memory.loc.gov/)
SFU Media Lab (http://www.sfu.ca/media-lab/)

The influence of transcoding on media is evident in our culture (Matrix http://www.apple.com/trailers/wb/matrix_reloaded.html)

The process of moving from a site of inquiry to the construction of knowledge using new media is:

Reality > Media (capture, collection) > Digitization (data creation) > Database (data is cleaned up, organized, indexed) > Analysis (learning)

Contemporary software applications can be seen as database navigation tools. Programs like Flash, Director, Premiere, Final Cut Pro, ProTools, etc., create a narrative by allowing a particular trajectory to be defined by the user, through a database of stock material such as video, audio, buttons, objects, behaviors, etc.

Towards a method for using new media in research

(Specific articles on using video in research:

http://www.sfu.ca/media-lab/cmns362/Spotlight45.pdf

http://www.sfu.ca/media-lab/cmns362/spotlight72.pdf)

Before you begin:

What is the purpose of the research?
Is recording primarily for presentation or analysis?
What is the specific value of recording video, audio, or stills in context?
Is specific information required that cannot be obtained from another course?
Determine the difficulties using recording equipment in the particular context.
How will your recording compliment other methods of data collection?
Scout location to prepare for issues concerning accessibility, power, space, light, sound.
Create an equipment list, and check all equipment before recording.
Bring lots of writing implements and notebooks.

The Dry Run:

Always try out the equipment in a Dry Run before going in for actual data collection.
Adjust the placements of equipment, length of sessions, setup times, equipment list, etc. based on the Dry Run.

Reality:

Most situations require permission from your subjects before you start recording (a sample Talent Release Form is at:

http://www.sfu.ca/media-lab/cmns362/Talent%20Release%20Form.htm

http://www.qualidata.essex.ac.uk/about/document.asp)

The introduction of a recording device (camera, microphone, video camera) into a social situation invariably causes discomfort and possibly effects behavior (The Researcher Effect). To address this you can try:

Using a hidden recording device (caution: moral and ethical issues)
Introducing the recording device before you start recording (hours, days) to allow subjects to get familiar with it.
Having the subjects do the recording (participant research)

Media:

The more equipment you use the more it will effect the situation, sound and light quality may have to be sacrificed to reduce equipment requirements.
If you are capturing only for analysis and not for presentation, quality can be reduced but you must maintain intelligibility throughout or data will be lost.
External microphones are usually a good idea to be able to isolate required audio.
Background light and sound can interfere with a recording, but you must balance artifice with recording quality – your task is to capture reality not make a film.
Bring lots of extra tape and batteries
Label all tapes with:

Reel number (a specific number for each tape)
Research Project
Date and Time

Keep extensive field notes of details and observations (resist the inclination that you will remember, write it down). Write down what the recording device does not capture.

Digitization:

Decide on the format you will analyze your data (resolution, frame rate, channels, compression algorithm, etc.)
Calculate the required disk space (i.e., 3.5min of DV video = 1gig disk space) and make sure there is enough for the project.
Preliminary logging can happen while capturing data to the computer.
Future development will probably eliminate this stage, which takes data captured on a linear format (audio and video tape) and converts it to a non-linear form on the computer.

Database:

Once the data is captured (digitized) it must be cleaned up or removed of any information that is of no use to the inquiry (i.e. test shots, mistakes, shots without permission, etc.). Caution: careful what you through away, mistakes can be informative.
First stage of annotation is indexing data by date, subject, time, location, event, etc.
Second stage of annotation breaks down the start and end of specific events by key framing time-code locations. For example subject begins to speak about a subject at point A and stops at point B.
Third stage of annotation extracts as many details as possible from the data, such as transcriptions of an interview, emotional responses, specific key events, and all other observations
All annotations are sorted and indexed into logical orders based on your analysis protocol.

Analysis:

To fully log, transcribe, annotate, and analyze one hour of videotape takes about 20 hours.
Stimulated recall sessions are an option (selected parts of the recordings are played back to participants who are then asked to comment).
New media data can be analyzed using quantitative methods such as spectral analysis, event counts, etc., but it is more suited to a combination of techniques that include ethnomethodological approaches.
Audio and video provide rich data, which can be overwhelming if a strategy is not adhered to. Analysis strategies can effect how the previous stages of data gathering are done so this whole process benefits from repeated iterations.
Human behavior patterns within a specific contextual interaction require analysis that informs all levels of the inquiry.
Examples of ethnographic approaches to contextual research are:

The sonic portrait (R.Murray Schafer, 1977), defining acoustic environment by the production of a soundscape that is created from a listeners perspective.
Acoustic Communication Model (Barry Truax, 2001), combinations of objective features with subjective reactions create a milieu.
Readymade video for daily learning in an intensive care unit, (http://space.interactiveinstitute.se/projects/iva-projektet/concept.htm)