Oct 2009 - this document describes an experimental video system that is under development. I am hard at work on a demo as you read this. The ideas presented here grew out of work I did on Cuts but share no technology or methods with it.

pv is a video presentation and authoring system that allows the content and duration of video to be flexible, and customized on the fly according to specific viewing desires or personal profiles.

Basic features

Advanced features

pv allows a unique experience for each viewing, which can be personalized or changed as desired to fit a specific duration or set of content preferences. All experiences are different just as all people are different. Even with a single version of a traditional linear film, everyone experiences the movie differently, despite what the director may have intended. pv just lets us explore and exaggerate these differences for artistic reasons and personal entertainment.

pv brings the real-time features of non-linear video editing systems to the media player by compositing the final video presentation from multiple layerd video soures, but it allows high level real-time control over the details of how those sources are combined to create the seamless final product. It fits the indivdual editing decisions into a cohesive big picture.

The Big Picture: expand and contract duration, adjust content

Though we often think of a movie as a single entity, in reality there are many different versions of movies that people care about: the theatrical release, the extended edition, the director's cut, the deleted scenes and alternate takes on the DVD, foreign language editions, subtitles, commentary audio tracks, the trailer with those scenes that somehow don't appear in the actual movie, the TV version, the airplane verison, and on and on.

hierarchical video

Currently, video production employs complex non-linear editing systems to carefully arrange segments of video on multiple tracks, which is composited and rendered to produce the final output: a linear sequence of frames.

pv is a video player and editor, and keeps as much of the editing details in the final product as possible. A pv movie has metadata that describes the content according to a wide variety of metrics - and how these metrics fit together.

metadata

A key metric is plot. We can measure the relevance of any moment in the movie to the plot; we start with the idea of key plot points or developments and call these critical to the story. The importance of the rest of the movie varies by degrees. There are certainly momemnts that have no impact on the plot whatsoever. Considering that a typical DVD has deleted scenes, which we might say they have even less - perhaps they detract from the experience in some way like pacing.

Metadata for eaoch metric forms a curve over the duration of the movie. To shorten the duration of the movie while retaining key plot elements of the story, we set a viewing threshold - the minimum amount of plot we're willing to accept. The "normal" release of the movie has the threshold set at zero, in which 100% of the movie lies above it. If we're really short on time, perhaps we'd set it at 80%, in which case only 20% of the movie's plot points would be shown.

It's the role of the editor to draw this curve appropriately so that the movie "gracefully degrades" as less and less of it is dialed down with this method.

Other metrics

It's easy to see that the plot metric above is not unique. Although we've called it "the plot" no special knowledge of the plot is required. It just a value the editor has ascribed to the movie which changes over the duration. As such, many metrics can also be described for the movie. Some examples:

In sum, a metric is a percentage value which changes over the duration of the movie. At any point in the movie, the value describes how much the metric applies.

To customize your viewing experience, you simply set a desired window - maximum and minimum - for each metric. To see everything, you set the minimum to zero, the maximum to 100%.

Redefined: Fast Forward and Rewind

Thresholds can be changed at any time during viewing. Want to skip past the violent scene you're watching? Simply lower the maximum threshold for violence. pv will jump to the next point where the amount of violence falls within your acceptable viewing range. Unlike fast forward, there is no chance of going too far as you've isolated the content that you're interested in viewing.

Redefined: Scenes

This generalizes the notion of a "scene". In a movie that is of no fixed length or content, we can see an overview of the movie based on all the metrics it has been marked.

Intersection and Overlapping of Metrics

Metrics are interesting in isolation, but more interesting and useful when combined. Once we have a movie marked up, we can make complex viewing requests to movies.

Personal profiles give a personal experience

Profiles are a convenient way for individuals to describe their viewing preferences, limitations, audience or context.

For instance, kids can't watch the scary parts, nudity, violence, etc.: basically, MPAA PG or G rating. The difference with pf, though, is the same movie can suit many audiences.

Imagine a movie that can be both the equivalent of today's R and PG MPAA ratings. How could this work? pv could have the R-rated material outside of the PG range in a set of MPAA-equivalent metrics. To ensure this works, viewers would have a PG or an R profile which simply specifies appropriate range. pv would require an adult profile for pv to show content from the R range.

This leads us directly to measurement and comparison.

Measurement, comparison and search through existing video

Once movies have their pv metadata, we can measure compare and search. If we have a reasonable agreement of what violence means, the fact that a movie has a violence metric curve means we can attach a number to how much violence is in the movie simply by the integral of (area under) the curve.

Instead of having a panel decide whether a movie is R, we can see what its content actually is according to various metrics. We can see how much of a movie is violent, for instance, and compare it against other movies.

For movies which we like, we can find similar movies simply by comparing metrics.

New creative tools

in progress

pv is the successor to Cuts and fixes a conceptual problem with that system.

A cutfile described how content was to be played - basically scripting existing video. But there were many cases where one wanted to expand or contract the edit details in the cutfile.

For example, say a cut eliminated a scary part for the kids. Someone else made a slightly different cutfile for the same purpose, and others even more. There's no way to reconcile all these different variations on the same idea, and no way to trust them. We'd be crushed under massive cut proliferation.

pv solves this problem by letting an editor - any editor - create metadata for the movie describing the content. pv creates the presentation by matching the viewing profile with the content descriptions.

So if the movie has a scene which is marked as really scary, and your viewing profile is set to only moderately scary, the very scary scene will be skipped.

The magic of pv lies in how that is handled.

Russell Holt