6.2. Working with Camera Input

6.2.1. Getting Camera Input

Getting camera input is practically identical to getting microphone input. Use the static Camera.get( ) method in order to create a new Camera object that connects to a specific video device on the computer:

var user_cam = Camera.get();

As with Microphone.get( ), the Camera.get( ) method uses the default video device if no parameter is specified. However, you can also pass the method an integer value representing a different device. The indexes you can use with get( ) represent elements of the Camera.names array, which contains a list of the video input devices detected by Flash. Ideally, however, the user will select the default video input device using the Camera tab of the Macromedia Flash Player Settings panel from Figure 6-2.

Regardless, once you've created a Camera instance, you can retrieve the index and name of the device to which it connects using the index and name properties.

A video device may already be locked exclusively for use with another application. In that case, the Flash Player will not be able to make a connection to the device. When that occurs, Flash does not provide you with any explicit notification that the connection was unsuccessful. It will still return a new Camera object when you call the get( ) method. Programmatically it will appear as though everything worked just fine. If you want to be able to detect that a connection was made successfully, you can use the currentFps property. While it is not foolproof, a currentFps property of 0 usually indicates that Flash was unable to connect to the device. In that case, you may want to ask the user to check the device and close any other application monopolizing it.

6.2.2. Understanding Camera Permissions

Camera permissions work just like microphone permissions. When you attempt to use the video input via a Camera object, Flash checks whether the user has granted permission. If the user didn't previously check the Remember box in the Settings panel, the panel will open, prompting the user to allow or deny access. When the user selects an option, the Camera.onStatus( ) method is called. Otherwise, if the user has selected the Remember checkbox previously, you'll need to use the Camera.muted property to determine whether the user has granted permission. See "Detecting the Microphone Privacy Setting" for more details. The API and procedure for camera and microphone permissions are identical. Whenever the privacy setting changes, onStatus( ) is invoked on all Camera and Microphone objects that define such a handler, and the muted property is updated for each object as well.

6.2.3. Attaching and Publishing Video

Once you've created a Camera object, you can attach the input to a local Video object (so the user can monitor it) and/or publish the input to a FlashCom application via a NetStream object (so that other users can receive it).

If you want to attach the input to a Video object:

The following code attaches the input from a Camera object named user_cam to a Video object named content_vid:

content_vid.attachVideo(user_cam);

If you want to publish the camera input to a FlashCom application, attach the input to a stream using NetStream.attachVideo( ) and publish the stream using NetStream.publish( ):

publisher_ns.attachVideo(user_cam);
publisher_ns.publish("cameraStream", "record");

Unlike audio, video is not "silent" even if the motion level falls below a certain threshold. So publishing video to a stream will necessarily use some bandwidth, even if there's no motion. However, the bandwidth requirement is low enough to be considered negligible in most cases.

6.2.4. Adjusting Camera Settings

Even more so than audio input, video input often requires that you optimize some settings. The following sections explain how to programmatically adjust a Camera object to affect video quality, dimension, frame rate, keyframe interval, and camera responsiveness.

6.2.4.1 Working with modes

Most cameras are capable of varying modes distinguished by different widths, heights, and frame rates. By default, Flash uses dimensions of 160 x 120 pixels and 15 frames per second (fps) when connecting to a camera. However, Camera.setMode( ) specifies the camera mode programmatically.

The setMode( ) method requires at least three parameters to specify the requested camera mode: the width, height, and frame rate. For example, the following code instructs Flash to attempt to set the mode of the camera associated with user_cam to 320 x 240 at 30 fps:

user_cam.setMode(320, 240, 30);

While setMode( ) may seem fairly unassuming, each camera is preconfigured with a set of modes at which it can operate, and different cameras may support different modes. Therefore, when you set a camera's mode to 320 x 240 at 30 fps, the actual result may differ slightly (or drastically). In order to predict what kinds of behaviors setMode( ) can produce, it can be helpful to understand the logic that it uses. That said, setMode( ) attempts to make sensible choices so that ActionScript programmers don't have to worry about the details.

The setMode( ) method follows some general rules. It won't set the camera to a mode that exceeds the parameters passed to it. However, it will, if necessary, use a mode with smaller dimensions and/or a lower frame rate. The algorithms that the method uses are fairly complex. In brief, Flash compares the area of the requested dimensions with the area of each of the native modes, selecting the best match available. If the selected mode does not match the aspect ratio, the video is cropped. If the selected mode does not match the frame rate, frames are dropped.

The important point is that you won't get exactly the mode you ask for if the camera doesn't support it.

By default, the frame rate is given priority in finding an appropriate mode. You can give priority to the area calculation by passing true as the fourth, optional parameter to setMode( ). Let's look at a few example scenarios in order to see how setMode( ) works.

In some cases, selecting whether the area or frame rate gets priority simply doesn't matter. Consider the following setMode( ) call, which attempts to set the camera to 320 x 240 at 30 fps:

user_cam.setMode(320, 240, 30, true);

Now consider a camera that supports the following native modes (sorted by fps, then by area in descending order):

176 x 144 @ 30 fps

160 x 120 @ 30 fps

128 x 96 @ 30 fps

80 x 60 @ 30 fps

320 x 288 @ 25 fps

320 x 240 @ 25 fps

352 x 288 @ 15 fps

640 x 480 @ 13 fps

512 x 288 @ 13 fps

The preceding setMode( ) command selects the 320 x 240 at 25 fps mode because it is an identical area match and also a close match for the frame rate.

In this example scenario, even if frame rate is given priority (as indicated by omitting the fourth parameter or passing false):

user_cam.setMode(320, 240, 30, false);

the same final native mode is selected, because the available 30 fps native modes support only areas that are less than the requested dimensions.

However, sometimes the area/frame rate priority setting can have an effect, even a drastic one. The following example assumes the camera has the same native modes as in the preceding example. However, in this case, setMode( ) is called to specify dimensions of 640 x 480 at 30 fps. If area is given priority:

user_cam.setMode(640, 480, 30, true);

the chosen mode is 640 x 480 at 13 fps because it is the best match for the area. When area is given priority, if the Camera class is unable to match the dimensions exactly, it will retrieve the next largest settings and then crop the image (unless the requested dimensions exceed those supported by the camera).

However, if frame rate is given priority:

user_cam.setMode(640, 480, 30, true);

the selection is 320 x 288 at 25 fpsvery different indeed! Because the available 30 fps native modes were too small an area compared to the requested 640 x 480, setMode( ) chose the mode with the biggest area that supported a high frame rate (in this case 25 fps is reasonably close to 30 fps).

The preceding scenarios are only two of many possible ones. For complete details on the algorithm used to select the native mode that best matches the requested parameters, see:

http://flash-communications.net/technotes/setMode/index.html

You can retrieve the current mode's settings using the read-only width, height, and fps properties of a Camera object. The fps property determines the maximum frame rate at which Flash will capture video using the current device settings. However, it does not necessarily mean that it will always capture at that rate. If necessary, Flash will drop the frame rate in order to be able to continue to publish a stream given other settings and the amount of bandwidth available. You can retrieve the actual frame rate at any given point by reading the value from the read-only currentFps property.

One logical approach might be to query the camera to obtain a list of native modes, display them in a listbox, and prompt the user to select a supported mode. Unfortunately, the ActionScript API does not provide a method to obtain a list of native modes supported by a camera (whatever method Flash uses internally to determine the available modes is not exposed to ActionScript).

Professionals responsible for buying cameras and setting up high-quality conferencing and live event broadcasts should select cameras based, in part, on the available modes. If you have a camera with known modes, script your application to selector allow an operator to selectfrom a list of modes appropriate for the event. If you don't have control over the equipment, in which case the camera's native modes are unknown, consult the URL cited earlier to help determine how to select available modes with reasonable bandwidth requirements.

You can publish only one size stream from a web cam at a time. If you create two separate instances with Camera.get( ) that refer to the same video source, changes made via setMode( ) to either instance are applied to both instances. To publish two different bit rates of the "same" footage, you need two capture sources. (Likewise, to capture audio input at different sampling rates simultaneously, you need two separate microphones.)

There are hardware and software solutions that make a single camera appear like multiple devices to the Flash Player, such as ViewCast's SimulStream for Osprey Video cards:

http://www.viewcast.com/simulstream_main.html

6.2.4.2 Working with quality

Video data can require a lot of bandwidth, which depends on the video's dimensions, frame rate, and quality (amount of compression). If your FlashCom application is running on a high-speed intranet in which bandwidth is not an issue, you can use higher-quality settings. But even in such cases, there are limits to what a PC can reasonably capture; for example, you shouldn't exceed 80% of CPU capacity on Windows.

However, most applications are deployed over the Internet, an environment in which the bandwidth requirements profoundly affect the application's functioning and the user experience. Therefore, you must optimize the video quality given the bandwidth of the user's connection. Using setMode( ) to select the smallest appropriate dimensions and lowest acceptable frame rate will assist you in managing bandwidth considerations. Additionally, the quality setting of the video plays a major role, so you can further tune the bandwidth requirements of the video using the Camera.setQuality( ) method.

The setQuality( ) method accepts two parameters that indicate the trade-off between data rate and appearance quality: an integer indicating the number of bytes per second to which the video data should be compressed and an integer from 0 to 100 indicating the quality of the video. The three basic scenarios when using setQuality( ) are:

Bandwidth is the priority

Specify the bytes per second value and set the second parameter to 0. Flash will reduce the quality (increase compression) to keep the video within the specified bandwidth.

Quality of the video image is the priority

Specify 0 for the first parameter; the higher the second parameter (from 0 to 100), the higher the image quality. If necessary, Flash will drop motion quality and the frame rate in order to maintain the image quality of the video.

Quality and bandwidth are equally important

Specify non-zero values for both parameters. Flash will try to compress the video within the quality and bandwidth settings. If necessary, Flash will drop the frame rate in order to maintain the bandwidth and quality.

Table 6-1 gives you some general starting points when using setQuality( ). Macromedia uses 16,000 bytes/sec (about 16 KB/s) for the default video bandwidth over a modem. Although a dial-up modem cannot handle 16 KB/s, Flash dynamically adjusts the frame rate as necessary.

Table 6-1. Suggested bandwidth and image quality settings for video over various connection types

Connection	Prioritize bandwidth	Prioritize image quality
56K modem	setQuality(4000, 0)	setQuality(0, 65)
DSL/cable	setQuality(120000, 0)	setQuality(0, 90)
LAN	setQuality(400000, 0)	setQuality(0, 100)

The effects of setQuality( ) are visible only in the compressed version of the video. The video that is published over a NetStream object is compressed, so any subscriber to that stream will see the effect of the setQuality( ) settings. However, if you attach video input directly to a local Video object, the compression will not be visible by default. Regardless of how much compression you apply, the video will still appear as crisp and smooth as the device can capture. To view the compressed video within the same client movie that is capturing the video, use setLoopback(true) to emulate the compression (passing false indicates no compression locally):

user_cam.setLoopback(true);

Using loopback is processor-intensive because it requires the Player to compress the video and then decompress it for display. Therefore, the loopback mode is typically used only during the development and testing phases.

The following code allows you to see the effects of setQuality( ). You can include it on the main timeline of a Flash file in which you have placed a Video object named content_vid and NumericStepper components named cnsQuality and cnsBandwidth on stage:

var user_cam = Camera.get(  );
// Display the video in the Video object.
content_vid.attachVideo(user_cam);
// Set the loopback so that you can view the compressed video.
user_cam.setLoopback(true);
var qualityListener = new Object( );
qualityListener.change = function (event) {
  user_cam.setQuality(user_cam.bandwidth, event.target.value);
};
cnsQuality.addEventListener("change", qualityListener);
cnsQuality.minimum = 0;
cnsQuality.maximum = 100;
cnsQuality.value = user_cam.quality;
var bandwidthListener = new Object( );
bandwidthListener.change = function (event) {
  user_cam.setQuality(event.target.value, user_cam.quality);
};
cnsBandwidth.addEventListener("change", bandwidthListener);
cnsBandwidth.minimum = 0;
cnsBandwidth.maximum = 400000;
cnsBandwidth.stepSize = 1000;
cnsBandwidth.value = user_cam.quality;

Test the movie and adjust the quality and bandwidth sliders to see their effect on the video.

6.2.4.3 Working with keyframes

Video in Flash uses keyframes in a way that is different from the way the Flash timeline uses keyframes. Depending on the bandwidth available and setMode( ) and setQuality( ) settings, Flash compresses video data within each 8 x 8 pixel block of each video frame. High levels of compression produce images with noticeable "blockiness" as a result. However, to reduce bandwidth consumption further, Flash does not always send complete frames. Frames for which complete image information is sent are called keyframes. The frames in between keyframes contain image data for only regions in which changes have occurred. The keyframes (called delta or difference frames) contain information about every pixel of the image regardless of whether or not anything has changed since the previous keyframe. Keep in mind, however, that a keyframe's quality may still be low, even though it contains complete image information. The amount of compression applied to the video still affects the quality of keyframes.

By default, Flash video is published with a keyframe every 15 frames. However, you can adjust the keyframe interval (from 1 to 48) to better suit your application and audience. Setting the value to 1 means that each frame is a keyframein other words, the data for each frame is complete. This dramatically increases the bandwidth and workstation processor requirements. A higher value for the keyframe interval generally means that the bandwidth requirements are decreased. There is no loss of video quality introduced by reducing the number of keyframes. However, if the video frame's contents are changing rapidly, more frequent keyframes can actually reduce the overall bandwidth requirements by reducing the size of each delta frame. That is, increasing the keyframe interval saves bandwidth most effectively when the contents of the frame aren't changing very much. Fortunately, this applies to most FlashCom applications, in which the video is often a talking head (however, it works best if the subject doesn't move around a lot).

You can set the keyframe interval using the setKeyFrameInterval( ) method. Simply call the method and pass it an integer value from 1 to 48:

user_cam.setKeyFrameInterval(10);

You can retrieve the current keyframe interval using the read-only keyFrameInterval property.

Understanding Video Objects When you want to display Flash video via a Camera object or a NetStream object, you need to use a Video object. There is no programmatic way to directly create a Video object. Instead, you have to create a Video symbol in the Library and add an instance of that symbol to the Stage. Furthermore, even after you have created a Video symbol in the Library, there is no way to directly attach an instance to the Stage programmatically. However, you can work around the issue by creating a movie clip symbol that contains a Video object. Movie clip symbols can be attached to the Stage programmatically if you set them to export for ActionScript in the symbol's Linkage Settings dialog box in the Library. Therefore, if you create a movie clip symbol containing a Video object, you can programmatically use it to add a Video object to the Stage at runtime. Although a Video object has properties such as _width and _height, you should place it inside a movie clip for greater programmatic control, because the MovieClip class supports additional properties and methods.

6.2.4.4 Working with activityLevel

Just as you can programmatically work with the sound activity level for a Microphone object, you can perform analogous operations with the motion activity level of a Camera object. The main difference is the slight variations in the names of properties and methods used.

To read the current activity level of a Camera object, use the read-only activityLevel property, just as you would with a Microphone object. The range of values for activityLevel is from 0 to 100.

The Camera.setMotionLevel( ) method sets the threshold at which to notify your application of motion. The setMotionLevel( ) method requires at least one parameteran integer from 0 to 100indicating the minimum motion level required for the Camera object to be activated. The default value is 50. You can optionally specify a second parameter that indicates the number of milliseconds before the object times out after the activity level falls below the minimum. The default value is 2000 milliseconds (2 seconds). This sets the motion level threshold to 35 and sets the timeout period to 1 second of inactivity:

user_cam.setMotionLevel(35, 1000);

You can retrieve the current minimum motion level and timeout setting with the read-only motionLevel and motionTimeOut properties.

Although Camera.setMotionLevel( ) is analogous to Microphone.setSilenceLevel( ), there is at least one major distinction due to the differences in how video and audio are treated. When audio and video are being published, the microphone or camera has to capture data to determine if the activity threshold is reached. If the threshold specified by Microphone.setSilenceLevel( ) is not reached, the application does not transmit the audio to FlashCom. (That is, the data is discarded if the input is deemed "silent.") However, even if the video capture is not changing very much (low motion), the video is still transmitted to keep the picture up-to-date.

Therefore, Camera.setMotionLevel( ) specifies the value at which the Camera.onActivity( ) event handler should be called, not whether the video data should be transmitted. The onActivity( ) method of a Camera object works just like the corresponding method of a Microphone object. The method is passed a Boolean parameter indicating whether the object has been activated (true) or deactivated (false):

user_cam.onActivity = function (activated) {
  trace("Is the camera activated? " + activated);
};

If you want to literally stop transmitting data when activity is low, you can unpublish the stream in the onActivity( ) handler. However, low-motion video (in which the picture is not changing much and the video has very small delta frames) with a long keyframe interval uses very little bandwidth (though clearly not zero, like audio silence).

One prudent use of an onActivity( ) handler is to determine if no one is standing in front of a camera-enabled kiosk. If the program detects no motion activity, you might cycle back to an attract loop. In this case, you should use setMotionLevel( ) to set the motion threshold and timeout period as appropriate for your kiosk environment.