Dr Dave

I've had a lot of requests to share my physics-enabled, WPF layout control for Microsoft Surface. This was demonstrated in the previous series of posts, Surface Physics Demo Part 1, Part 2, and Part 3.

So here it is! The following downloads are available:

1. Physics Library (binary), .dll (zip'd), 19Kb. The physics library and layout control.
2. Surface Physics Sample (install), .msi (zip'd), 1,046Kb. The sample application for demonstrating the physics library and layout control.
3. Readme for Surface Physics Sample, .pdf, 1,626Kb. Readme for the sample application.
4. Surface Physics Sample (source code), Visual Studio 2008 Project (zip'd), 907Kb. Source code for the the sample application.

Note that the Physics Library itself is currenly only available as a binary.

The Surface Physics Sample demonstrates many of the features supported by this library. However, to get you started with using the library in your own projects, I'll discuss how to enable basic physics for a simple ScatterView sample.

Migrating ScatterView to PhysicsView

This example demonstrates how to take a simple ScatterView sample and migrate it to make use of the physics-enabled layout control. You'll need the Microsft Surface SDK, available from the MSDN site here, and access to a Microsoft Surface or at least the Surface Simulator in the SDK. You'll also need the physics library.

First create a new Surface Application (WPF) project using the Visual Studio template inlcuded in the SDK. In SurfaceWindow1.xaml add a ScatterView to the default Grid control as follows:

<s:ScatterView Name="ScatterView1">
    <s:ScatterView.ItemTemplate>
        <DataTemplate>
            <Border BorderBrush="White" BorderThickness="2">
                <Image Source="{Binding}" Width="128" Height="96" />
            </Border>
        </DataTemplate>
    </s:ScatterView.ItemTemplate>
</s:ScatterView>

In SurfaceWindow1.xaml.cs add an event handler for the Loaded event and add some items to the layout control:

public SurfaceWindow1()
{
     InitializeComponent();

     // Add handlers for Application activation events
     AddActivationHandlers();

     this.Loaded += new RoutedEventHandler(SurfaceWindow1_Loaded);
}

void SurfaceWindow1_Loaded(object sender, RoutedEventArgs e)
{
    ScatterView1.ItemsSource = System.IO.Directory.GetFiles(@"C:\Users\Public\Pictures\Sample Pictures");
}

Run the project and if you're running the Surface Simulator you should see the following:

Figure 1. Simple ScatterView example. Note that the blank image is from a hidden file (desktop.ini) in the Sample Pictures folder.

We'll use this as a baseline to migrate from ScatterView to PhysicsView. First of all, copy the physics library (Physics.SurfaceControls.dll) into the project, and add a reference to it. Then add a namespace declaration at the top of SurfaceWindow1.xaml as follows:

xmlns:p="clr-namespace:Physics.SurfaceControls;assembly=Physics.SurfaceControls"

In SurfaceWindow1.xaml do a find and replace on s:ScatterView with p:PhysicsView and change the name from ScatterView1 to PhysicsView1. The rest of the markup remains unchanged, and the layout control should now look like this:

<p:PhysicsView Name="PhysicsView1">
    <p:PhysicsView.ItemTemplate>
        <DataTemplate>
            <Border BorderBrush="White" BorderThickness="2">
                <Image Source="{Binding}" Width="128" Height="96" />
            </Border>
        </DataTemplate>
    </p:PhysicsView.ItemTemplate>
</p:PhysicsView>

We can now set the ItemsSource on the new layout control. The next thing we need to do is inform the physics library of the physical properties of these items. Before we do this, however, we should set up some walls defining the bounding area. The library doesn't make any assumptions here since this area doesn't have to be rectangular nor aligned with the x,y axes. In this case we'll simply add four walls, inset from the extent of the screen by 16px (remembering that a Surface application runs at 1024 x 768px). Update SurfaceWindow1_Loaded to look like this:

void SurfaceWindow1_Loaded(object sender, RoutedEventArgs e)
{
    //ScatterView1.ItemsSource = System.IO.Directory.GetFiles(@"C:\Users\Public\Pictures\Sample Pictures");

    // add walls
    PhysicsView1.Bounds = GenerateWalls(0.5, new Rect(new Point(16, 16), new Point(1008,752)));

    // set data context
    PhysicsView1.ItemsSource = System.IO.Directory.GetFiles(@"C:\Users\Public\Pictures\Sample Pictures");

    // add behaviours
    AddBehaviours(this.PhysicsView1);
}

The methods for setting the bounds and item properties are as follows:

List<WallBody> GenerateWalls(double restitution, Rect rect)
{
    return new List<WallBody> {
        new WallBody { Normal = new Vector(1, 0), StartPoint = rect.TopLeft,
                EndPoint = rect.BottomLeft, Restitution = restitution }, // left
        new WallBody { Normal = new Vector(0, 1), StartPoint = rect.TopLeft, 
                EndPoint = rect.TopRight, Restitution = restitution }, // top
        new WallBody { Normal = new Vector(-1, 0), StartPoint = rect.TopRight,
                EndPoint = rect.BottomRight, Restitution = restitution },  // right
        new WallBody { Normal = new Vector(0, -1), StartPoint = rect.BottomLeft,
                EndPoint = rect.BottomRight, Restitution = restitution }}; // bottom
}

void AddBehaviours(PhysicsView physicsView)
{
    Random random = new Random();

    // ensure layout
    physicsView.UpdateLayout();

    for (int i = 0; i < physicsView.Items.Count; i++)
    {
        // get item
        PhysicsViewItem item = physicsView.ItemContainerGenerator.ContainerFromIndex(i) as PhysicsViewItem;

        // set properties
        Body body = new RectangularBody
        {
            Width = item.DesiredSize.Width,
            Height = item.DesiredSize.Height,
            Density = 0.01,
            InertiaConstant = 0.5,
            Restitution = 0.5,
            Orientation = random.NextDouble() * Math.PI * 2,
            Location = new Point(random.NextDouble() * physicsView.ActualWidth,
                random.NextDouble() * physicsView.ActualHeight),
        };

        // add item
        PhysicsCanvas.SetChildBody(item, body);
    }
}

That's it! The items will now collide with one another. Run the project and you should now see the following:

Figure 2. PhysicsView Items now collide with one another and do not overlap.

Regrettably I've had no time to extend this work, so a lot of features remain un-implemented at this time. Examples inlcude multi-touch manipulations on the individual items themselves, a better dampening algorithm etc. However, I hope it may still prove useful in some cases.

The source code for this example can be downloaded here.

In Part 1 and Part 2 I focussed on 2D features. This makes a lot of sense for Surface applications, as fundamentally items move in two dimensions, however there are particular scenarios that lend themselves to 3D, one of which I'll describe later in this post.

Video 1. Surface physics demo.

The video is divided into the following sections:

Layout

In some cases it is desirable to arrange the items into preset patterns, for example as part of an "Attract Mode" application, or when interacting with physical objects placed on the Surface. This screen defines some basic patterns and "locks" the items to the nearest position in the pattern from their original location. Selecting an item releases the "lock".

3D Rolling Spheres

Spheres lend themselves to an intuitive motion on a 2D-plane, such as when playing marbles, pool etc. When a texture is added to the sphere, it is important to ensure that the sphere "rolls" correctly when moved. Several examples of textures are shown in the video.

Here are some further screenshots.

Figure 1. Marbles

Figure 2. Pool balls

In Part 1 I introduced a generic framework I have produced for a Surface-enabled WPF layout control which has basic physical interactions.

Aswell as demonstrating some additional physical behaviour, I wanted to focus this post on some Surface-specific features. One of several key tenets of Surface development is multidirectional applications. This is often overlooked, even when developing for Surface as the developer typically uses a standard development PC with a vertically-oriented screen. I should say that the radio buttons down either side of the demo aren't part of the framework I describe here - they are merely present to allow me to illustrate different features over several pages - so should be "ignored" when it comes to a discussion about multidirectional UI.

Let's jump straight into a video.

Video 1. Surface physics demo.

The video is divided into the following sections:

Further Materials

Other than adding some black and white "plastic" rounded tiles, I've included some "crystal" materials. These are 3D models of a typical facetted gem, with some suitable lighting and transparency. The colors are randomly generated each time the page is selected.

Spring Forces

I'd wanted to add these from the start, as they are great fun to play with. Whenever a "spring" tile (another poker chip) is placed in the Surface, any selected items are joined to it via a spring, or piece of elastic. Muliple springs can be connected to multiple objects. When combined with directional forces the springs will "swing" accordingly. A basic spring algorithm is used, with a configurable spring constant and length (quite "loose" and "short" respectively in this sample).

360° Directional Forces

This section illustrates how a "dial" object (you guessed it, another poker chip) can be used to control the direction of a force. When placed on the Surface, the current direction is indicated and can be changed by rotating the object.

360° Directional Lighting

In a similar approach to the directional forces above, a "dial" object is used to control the direction of the dominant light source in the model.

Here are some further screenshots.

Figure 1. Springs

Figure 2. Force direction

Figure 3. Gem lighting

Note that these screenshots are from the Surface Simulator, so the physical objects (i.e. spring object, force, and lighting dials) are necessarily virtual.

In the next article I'll discuss 3D features.

I've recently been doing some work on a physics engine sample for Microsoft Surface. The principal purpose of this work was to investigate how adding physical characteristics to virtual items adds to the realism of a Surface experience. The work had three main areas of focus:

1. To add physical behaviour to virtual items in a Surface experience
2. To combine physical behaviour with 3D WPF templates, textures and lighting sources
3. To demonstrate how the object recognition capabilities of Microsoft Surface can be used to interact with these items

In addition to these points, I wanted to provide this functionality within a WPF layout control closely analagous to the Surface ScatterView control. In this way, it should be relatively easy to swap out a ScatterView implementation for a physics-based alternative.

Here's a video to illustrate the key features for this post.

Video 1. Surface physics demo.

The video is divided into the following sections:

Basic Interactions

Basic collision detection & response between circles, rectangles, and polygons. Notice that one can hold onto a given shape and flick other items into it. The held shape is affected by the collision but "springs" back to place. The items are all set to be the same density, so the size is proportional to the mass. The walls are "soft" and result in springy collision, rather than a hard collision between objects.

Basic Forces

Illustration of directional forces (basic acceleration) and point forces (a gravity algorithm). Notice that point forces with low drag result in a chaotic rotational motion - heavier items accrete towards the gravity source, and lighter items rotate futher away. Multiple point forces can be added. Combinations of directional and point forces is also possible, e.g. "hanging" an object of suitable mass below a point force. Note also that the physical tags (poker chips) have a virtual presence and result in collisions.

3D Objects and Materials

This section illustrates some lighting and textures, e.g. "wooden" and "marble" tiles. The lighting is consistent between objects, in this case pointing from lower left to upper right with respect to the camera position. Patterns tiles have image textures generated in code. In this example they simply use black and white brushes, but one of the great things about WPF is that I could as easily use different image brushes and, for example, "inlay" different wood textures to form the same pattern.

More Interactions

This section illustrates the use of "fixed" objects, with and without directional forces (actually the "fixed" objects are just non-selectable items of very large mass). It also illustrates some of the object recognition of Microsoft Surface. Ordinary business cards placed on the Surface become part of the "virtual" world. One can also use other physical objects (e.g. brochures, hands etc) to "sweep" items.

Configuration and Debugging

This screen allows configuration of just some of the properties of the physical environment, such as "bounciness" (restitution) of items and walls, directional and angular drag, directional forcees, maximum directional and angular velocities etc. It was also an invaluable tool in debugging the build as it shows positions and vectors graphically.

I've included some screenshots below.

Figure 1. Miscellaneous shapes

Figure 2. Wood textures

Figure 3. Pattern textures

Figure 4. Visual Debugging

In the next article I'll describe some more physics-related and Surface-specific features of this framework.