Developing a Robust Data Driven UI Using WPF – The DataModel

.NET, Software Development, WPF March 24th, 2008

imageIn the first post in the series I gave an overview of the pattern we’ll be using.
This post will go deeper into the DataModel, as defined in the previous post:

The DataModel is defined exactly as the Model in MVC; it is the data or business logic that stores the state and does processing of the problem domain.
The DataModel abstracts expensive operations such as data fetching without blocking the UI thread. It can keep data “alive” fetching it periodically from source (example: stock ticket), merge information from several sources etc.
The DataModel is completely UI independent and pretty much straightforward to unit test.

The DataModel exposes data in a way that makes it easily consumable by WPF. As such, all if its public APIs, called by WPF for data-biding, must be called on the UI thread only. It must not block the UI thread because we want a robust functional UI so it usually performs operations on a background thread using the Dispatcher to send results back to the UI thread.

Therefore, the simplest DataModel implementation exposes several public Properties that expose data, implements INotifyPropertyChanged and/or INotifyCollectionChanged, and it abstracts the way information is fetched (using background threads to avoid blocking the UI thread when fetching the data is an expensive operation).

For two-way binding a commit and rollback mechanism, a dirty flag, etc.  We’ll get to that later on…

As the DataModel implementation needs to abstract expensive data fetching operations and work with multiple threads we need some basic understanding of WPF’s threading model before we look at the DataModel implementation…

WPF Threading Model – A Quick Overview

A typical WPF uses two threads:

  • Rendering thread – runs in the background and handles rendering
  • UI thread – Receive inputs, handles events, paints the screen and runs application code.

The UI thread queues work items in a Dispatcher object. The Dispatcher object selects work items on a priority basis and runs each one to completion.
Every UI thread must have at least one Dispatcher, and each Dispatcher can only use one thread to execute work items.

Therefore, in order to build responsive UI that doesn’t block the UI thread, the application has to maximize the Dispatcher’s throughput by keeping work items small as to minimize the time the Dispatcher spends on processing them – which keeps other work items waiting causing the UI to lag.

In order to perform expensive operations without blocking the UI thread we can use a separate thread that will run in the background, leaving the UI thread free to process items in the Dispatcher queue. When the background thread is done processing it can report results back to the UI thread for display.
Doing this isn’t trivial as Windows only allows UI elements to be accessed by the thread that created them. This means that the background thread we used for some long-running task cannot access and update our UI when it is finished (or during work to show progress) – a background thread updating a control (such as a list box) during its rendering can cause strange UI behaviors that this limitation is there to prevent.

WPF uses the following design to enforce this kind of coordination between the UI thread and other threads:
Most of the classes in WPF derive from DispatcherObject. During construction, a DispatcherObject stores a reference to the Dispatcher linked with the current running thread – creating an association between itself and the thread that created it.
At the beginning of every method in the DispatcherObject, it calls VerifyAccess which compares the Dispatcher associated with the current thread with the Dispatcher stored during the object’s construction – if they do not match it throws an exception.

If only the creator of a DispatcherObject can access it, how can a background thread interact with the user?
The background thread does not access the UI directly but it can ask the UI thread to perform a task on its behalf by registering work items to its Dispatcher using it’s Invoke (for a synchronous call that returns when the UI thread finished executing the delegate) or BeginInvoke methods (which runs asynchronously)

The DataModel Class

So now, after the brief discussion on the use of the Dispatcher we can start coding our base DataModel class.
We’ll start with the simple class and constructor definition:

public abstract class DataModel : DispatcherObject, INotifyPropertyChanged
{
    public DataModel()
    {
    }


We’re deriving from DispatcherObject because we need to have the Dispatcher available so that we can run background jobs that dispatch results to the UI thread.

As discussed earlier, each call to the DataModel should be made on the UI thread. Therefore we would like to enforce that limitation at the beginning of each publicly exposed API. The DispatcherObject class that we derived from contains a VerifyAccess() method that does just that. The method is public but unfortunately marked with the [EditorBrowsable(EditorBrowsableState.Never)] attributes which will make it hard to find for developers using driving their data model from our class.

To resolve this I simply defined a protected method as follows:

/// <summary>
/// Makes sure the call is in the correct thread (the UI thread) by comparing the current dispatcher
/// object with the dispatcher we got when the DataModel was created.
/// </summary>
[System.Diagnostics.Conditional("Debug")]
protected void VerifyCalledOnUIThread()
{
    this.VerifyAccess();
}

This method will be visible to anyone deriving from our class and it simply calls VerifyAccess to make sure code is made from the UI thread.
The Conditional attribute makes this code execute only in debug bits avoiding this kind of assertion on retail bits – some performance gain.

In order to support asynchronous data fetching the DataModel should encapsulate the information about its state – valid (data fetched), invalid (error fetching data), fetching (processing).

public enum DataModelState
{
    /// <summary>
    /// The model is fetching data
    /// </summary>
    Fetching,
    /// <summary>
    /// The model is in an invalid state
    /// </summary>
    Invalid,
    /// <summary>
    /// The model has fetched its data
    /// </summary>
    Active
}

The data model’s state is exposed using a property:

public DataModelState State
{
    get
    {
        VerifyCalledOnUIThread();
        return _state;
    }
    set
    {
        VerifyCalledOnUIThread();
        if (value != _state)
        {
            _state = value;
            OnPropertyChanged("State");
        }
    }
}

We also implement INotifyPropertyChanged to allow the model to communicate changes in its values.
Since adding\removing event handlers to the PropertyChanged event is a public API exposed by the DataModel, it also requires verification that calls to it are made from the UI thread. We’ll define our own add\remove handlers in order to perform this verification:

protected virtual void OnPropertyChanged(string propertyName)
{
    VerifyCalledOnUIThread();

    if (_propertyChangedEvent != null)
    {
        _propertyChangedEvent(this, new PropertyChangedEventArgs(propertyName));
    }
}

#region INotifyPropertyChanged Members
public event PropertyChangedEventHandler PropertyChanged
{
    add
    {
        VerifyCalledOnUIThread();
        _propertyChangedEvent += value;
    }
    remove
    {
        VerifyCalledOnUIThread();
        _propertyChangedEvent -= value;
    }
}
#endregion

Any property that we’ll add to our data model will call OnPropertyChanged on its setter in order to notify it has changed.

It’s Alive!

One more ability we’d like to add to our DataModel class is the ability to enable\disable it.
As defined earlier, the DataModel encapsulates the logic of fetching data and keeping it “alive” and up to date. To do that, it’ll need to keep some internal timer for updating information or register to some change notification event on its source.
This will keep the DataModel alive and can result in memory leaks, which is why we need some way to turn the DataModel on and off, allowing it to unregister from its data sources when that connection is no longer required:

public bool Enabled
{
    get
    {
        VerifyCalledOnUIThread();
        return _isEnabled;
    }
    set
    {
        VerifyCalledOnUIThread();
        if (value != _isEnabled)
        {
            _isEnabled = value;
            OnPropertyChanged("Enabled");
        }
    }
}

public void Enable()
{
    VerifyCalledOnUIThread();

    if (!_isEnabled)
    {
        this.Enabled = true;
        OnEnabled();
    }
}

public void Disable()
{
    VerifyCalledOnUIThread();

    if (_isEnabled)
    {
        this.Enabled = false;
        OnDisabled();
    }
}
protected virtual void OnEnabled()
{
}
protected virtual void OnDisabled()
{
}

When binding UI elements to the DataModel we’ll need some mechanism to enable the DataModel when the element is loaded and disable it when the element is unloaded. There’s an elegant way to implement this behavior which we’ll implement in a future post.

That’s it! We’ve got a basic class to derive out data models from. Note that we’re only addressing one-way data binding for the moment. We’ll address a two-way data model (which requires the ability to commit\rollback data etc.) in future post.

On the next post we’ll look into a concrete DataModel implementation for our Stocky application.

You can download the code for this post from here:

 

Further Reading

kick it on DotNetKicks.com

Tags: , , , ,

Developing a Robust Data Driven UI Using WPF – Introduction

.NET, Software Architecture, Software Development, WPF March 18th, 2008

WPF, Microsoft’s not-so-new-anymore UI technology offers new capabilities allowing both developers and designers to work together to achieve a stunning experience for their applications.

Power, however, does not come without complexity, and WPF does not provide a framework or a model to solve many of the problems faced by developers and designer when building an application:

1. Handling Rich Data Forms. Many applications, especially when it comes to enterprise applications, rely heavily on displaying and manipulating data. Fetching the data while keeping the UI alive and responsive is a complicated task that’s also hard to debug and requires an experienced developer doing the work.
Can we come up with a framework that will simplify data fetching?

2. Testability is a Requirement for Software Development Framework. Development organizations are no longer satisfied with simple reduction of costs for initial development and there’s a growing demand for frameworks and tool to facilitate a sustainable and agile development process.
Can we come up with a model that will allow writing tests for the application’s UI and behavior?

3. Metadata Driven User-Interface. WPF provides XAML as a meta-model for UI definitions. However there is no clear separation between metadata and code which is a mess when it comes to designer and developers working together.
Can we come up with a model to allow developers provide all the UI logic as closed building blocks that designer can just use in a plug-and-play manner?

Providing a Framework for Building Robust, Data-Driven UIs

The Model\View\Controller (MVC) architectural pattern has long been used by complex applications to present large amount of data to the user.
The pattern allows developers to separate the actual data (Model) from the user interface (View) and the business logic manipulating the data (Controller).

In the following set of articles I will present a variation of the MVC pattern tailored for modern UI development (in WPF) where we’d like the View to be the responsibility of a designer rather than a classic developer writing code.

I’ll be using the DataModel\ViewModel\View terminology to describe the pattern (although you may find the same pattern described using various other terminologies when browsing the net).

Introducing the DataModel\ViewModel\View Pattern

As mentioned earlier, the DataModel\ViewModel\View pattern is a variation of the MVC pattern. Its focus is on making the View, which is the actual UI presented to the user, the responsibility of a designer - a person who is generally more oriented towards graphics, art and interaction than to classic coding.

The design of the view should be done in a declarative form (XAML) using a WYSIWYG tool (Expression Blend).
In short, the actual UI is developed using different tools and languages by a person with a different skills set than business logic and data backend.

In order to understand the meaning behind the DataModel\ViewModel\View terminology lets look at the following diagram describing
typical architecture for our application’s presentation using this pattern:

image

The DataModel

The DataModel is defined exactly as the Model in MVC; it is the data or business logic that stores the state and does processing of the problem domain.
The DataModel abstracts expensive operations such as data fetching without blocking the UI thread. It can keep data “alive” fetching it periodically from source (example: stock ticket), merge information from several sources etc.
The DataModel is completely UI independent and pretty much straightforward to unit test.

The View

The View consists of visual elements and represents the actual user interface presented to the users (buttons, windows, graphics, etc.). It also defines interaction for keyboard shortcuts and other input devices .

The View is defined declaratively in XAML by the designer (usually using a tool such as Expression Blend).
Using such a declarative model makes it to harder to represent some state that the original  View from the MVC pattern was meant to deal with – this includes dealing with multiple modes of interaction (such as “view mode” and “edit mode”) that change the visuals and behavior of the controls.

This is where we make use of WPF’s advanced data binding mechanism. In a simple scenario we can simply bind the View to the DataModel and use binding expressions to perform one-way binding for display only values or two-way binding to allow editing values in the DataModel.

In most scenarios, however, only a small subset of the application’s UI can be bounded directly to the DataModel. This can be the case when the DataModel is a pre-existing class or data schema over which the application developer has no control. The values exposed by the DataModel are likely to require some processing in order to allow binding to UI elements. There may also be several complex operations that require code implementation and do not fit into the strict declarative-only definition for a View but are too application specific to be part of the DataModel (which we might not have control over).
We may also want to save some view state such as view mode (view\edit\etc.) or item selection etc.

To bridge this gap between the declarative View and the DataModel we define the ViewModel…

The ViewModel

The ViewModel bridges between the DataModel and the View and performs all the tasks mentioned in the previous paragraph.
The terms is meant to describe a “Model of a View” which basically means that the ViewModel abstracts all the behavior logic behind a specific screen (View) in the application.
The ViewModel include converters that can transform DataModel types into View types, Commands that can be executed the the View’s control and interact with the DataModel and general behaviors that can be attached to UI elements in the View.

Summary and Next Steps

stockyscreen

The DataModel\ViewModel\View defines a simple yet powerful pattern allowing developers and designers to collaborate on building a robust, data-driver WPF UIs.

It allows separating the data layer from the view layer and the UI to support easier development of granular components that are also unit-testable.

To demonstrate how the various pattern components are developed and used we’ll be going over the development process of a stock ticker widget-like application dubbed Stocky (screenshot on the right) and see how this development pattern simplifies the creation of an otherwise quite complicated little application.

References:

kick it on DotNetKicks.com

Comments (6) from www.ekampf.com/blog/:

Tuesday, March 18, 2008 4:09:58 PM (GMT Standard Time, UTC+00:00)

In my company We developing a very big medical system with UI based on WPF.
We used a combination of the Model-View-Presenter and the DataModel-View-ViewModel introduced by Den Crevier’s.
looking forward to see your implementation.

Ran Trifon

Tuesday, March 18, 2008 4:52:31 PM (GMT Standard Time, UTC+00:00)

Well it’s pretty much the same…
The goal here is to summarize all the information into one place. Dan’s post are pretty short and straightforwards aimed at experienced developers and these post are meant to be more detailed.
I am going to post about topics he didn’t mention though…

Eran Kampf

Tuesday, March 25, 2008 3:41:31 PM (GMT Standard Time, UTC+00:00)

Nice post – I’m really looking forward to seeing where you go with this. I’ve just recently being trying to find some guidance on setting up an MVC/MVP framework in WPF. Dan’s series is great but I must admit that I really didn’t understand it all until I began my own implementation and things began to “gel”. Will be great to see another perspective on it.

Nigel Spencer

Tuesday, March 25, 2008 3:53:55 PM (GMT Standard Time, UTC+00:00)

Thanks Nigel,
Next post in the series is already available at http://www.ekampf.com/blog/2008/03/24/DevelopingARobustDataDrivenUIUsingWPFTheDataModel.aspx

Eran Kampf

Tuesday, March 25, 2008 6:38:44 PM (GMT Standard Time, UTC+00:00)

Eran:
Just wanted to say “keep up the good work”. between your work and Dan’s series of articles, I think I’m starting to get a handle on this. My one request is that I’d like to see how your DataModel interacts with the DataAccess Layer against SQL Server. Maybe just something against Northwind. I realize this might be outside the main scope but I think it would be interesting.
Sincerely,
Dale Williams

Dale Williams

Tuesday, March 25, 2008 7:34:38 PM (GMT Standard Time, UTC+00:00)

Hey Dale,
Thanks for the feedback :)
The 3rd post in the series will show a concrete DataModel example.
Since I was aiming to show how I build a Yahoo finance widget clone I was building the DataModel on that – keeping a stock data up to date (kind of like in Dan’s article).
However, once you see how the DataModel fetching is implemented it doesn’t really matter if the actual data is fetched via SOAP call, http, or a DB access so you’ll be able to implement a one-way binding to a data source of your choice.
While the current implementation only deals with one-way binding (only fetching the data from the without the ability to update data on the source) I do plan to show how to implement two-way binding and support comitting and rolling back data in future posts.
Thanks,
Eran

Eran Kampf

Tags: , , ,

Duet Behind the Scenes: Bounded Items

Duet, SAP July 18th, 2007

Binding to Outlook item (Mail, Task, Contact, etc.) is one of the main features that differentiates Duet’s user experience from other OBA products such as Snap-Ins, Extensio etc. What exactly are bounded items? and how is binding implemented?

What are Bounded Items?

A bounded item is a simple Outlook item, which can be an email, appointment, contact etc. with an attached schema that defines it as a specific presentation object (presentation objects discussed in the previous post) and carries the business data related to that object. We call this a payload.

The payload is an encrypted XML containing the presentation object reference and business data mentioned above and attached to the Outlook item as a custom property.

For example, a payload attached to a contact item that represents a CRM customer would probably contain data such as the customer’s ID, company he works at, etc.

Before we examine how this payload is built, maintained and displayed to the user we first have to take a step back and understand how its schema defined. Or to be more accurate, how we define a presentation object.

Bound Item Definitions

If you remember the metadata illustration from the first post, one of the key parts in a presentation object’s metadata definition is its Data Definitions as displayed in the illustration:

 image

A presentation item’s Data Definitions contain the following information:

  1. Presentation object name- A unique resource name identifying this presentation object type
  2. Outlook definitions -Specifies the Outlook item type this objects relates to and its default folder location.
  3. Synchronization behavior – Type of synchronization used to synchronize to the backend.
  4. Business Data Schema – A schema for the business data this object carries with it
  5. Data Mapping to\from Outlook Properties – A set of mappings that define how the Outlook item’s default fields (Such as Subject, Location, Start\End Time) map to\from values in the item’s business data.

If we look at a definition for a CRM Customer object again, its name would be something like "urn:sap:duet:crm:customer" and its business data would probably contain name and contact information (address, phone numbers). This data would be mapped to the relevant Outlook fields (phone numbers to Contact item phone properties, address to the Contact item address property etc.) so that when the user edits the object the changes will be propagated back to the business data payload (and eventually to the backend).

This data definition, termed Bound Item Definition looks like this:

<BoundItemDefinition Name="urn:sap:duet:crm:customer" xmlns="http://schemas.microsoft.com/OBA/2005/BoundItemDefinition">
  <Outlook DefaultFolder="CRM Contacts" MessageClass="IPM.Contact" />
  <Synchronization>
    <Behavior>Always</Behavior>
  </Synchronization>
  <Properties>
    <Property Name="Crm.Customer.FirstName" Type="String">
      <Behavior>OutlookToXml</Behavior>
      <Outlook Name="FirstName" />
      <Xml XPath="/ns0:CRMContact/Name" Namespaces="ns0='urn:sap:duet:crm:customer'" />
    </Property> <Property Name="Crm.Customer.LastName" Type="String">
      <Behavior>OutlookToXml</Behavior>
      <Outlook Name="LastName" />
      <Xml XPath="/ns0:CRMContact/ns0:LastName" Namespaces="ns0='urn:sap:duet:crm:customer'" />
    </Property>
    ...
  </Properties>
  <XmlTemplate>
    <s0:CRMContact xmlns:s0="urn:sap:duet:crm:customer">
      <s0:FirstName>String</s0:FirstName> <s0:LastName>String</s0:LastName> <s0:FileAsName>String</s0:FileAsName> <s0:Comapany>String</s0:Comapany> <s0:JobTitle>String</s0:JobTitle>  ...
    </s0:CRMContact>
  </XmlTemplate>
</BoundItemDefinition>

Binding an Item

The process of binding an item is where we take a regular Outlook item and mark it has a presentation object. Basically we’re taking a regular Outlook item and marking it as a presentation object. The item’s behavior and schema for the business data attached to it are according to the presentation object’s data definitions. When binding an item the Duet platform adds 3 custom properties to the item:

  1. BoundItemId – A unique ID to represent this bounded item.
  2. BoundItemType – The type of presentation object that the item represents
  3. BoundData – The business data the item carries (payload).

In our example, a bounded IPM.Contact object representing a customer would have urn:sap:duet:crm:customerIn as its BoundItemType and the following XML as its BoundData:

<s0:CRMContact xmlns:s0="urn:sap:duet:crm:customer">
  <s0:FirstName>Eran</s0:FirstName>
  <s0:LastName>Kampf</s0:LastName>
  <s0:FileAsName>Kampf, Eran</s0:FileAsName>
  <s0:Comapany>SAP Labs Israel</s0:Comapany>
  ...
</s0:CRMContact>

The values in the BoundData are populated according to the data mapping rules specified in the bound item definition for our customer object.

Security Considerations

The BoundData can contain confidential information. Once retrieved from the backend systems we would like to limit access to it specifically to the user who got the information. Otherwise, sensitive information which is part of the user’s item can leek if, for example the user forwards the item to someone or if an IT administrator (or anyone else who has access to the user’s mailbox) looks at the item.

We use DPAPI to encrypt the BoundData using the current user’s credentials so that only the Duet user who received the data can decrypt it. This way, even if someone else gets hold of the BoundData value he has no way to decipher it…

This also means that when forwarding a Duet item the receiving side will not have Duet information available and will see the item as a regular unbounded Outlook item. I’ll discuss how forwarding of bounded items is performed in a future post dedicated to forwarding…

kick it on DotNetKicks.com

Tags: , , , , , , ,

Duet Behind the Scenes: Metadata Driven Solutions

Duet, SAP July 2nd, 2007

The Duet applications are built using metadata definitions that are interpreted during runtime and drive the different application’s workflow and UI.
How is Duet’s metadata defined and used? What are the advantages of this model? image

Presentation Objects

The Duet metadata defines business objects and how they interact with the user and the backend system. I like to refer to such objects as “presentations objects” because unlike the backend Business Objects that make sense to backend developers these are objects that make sense to the user.
For example, information about a person’s salary, compensation and HR information may be represented as several different business objects on the backend (or on several backend systems, maybe even from different vendors). The end-user, looking at a contact information on his Outlook doesn’t have to know about all the different backend and business objects that were used to aggregate the information on his screen.

Therefore, the Duet metadata defines such presentation objects - the information they carry, how this information is aggregated to and from the backend system(s)and how they interact with the user in his Office environment. So, from an abstract point of view, an application’s metadata structure is somewhat similar to this:

image

DuetML

All the UI components in Duet are defined by a markup language – DuetML. Each UI component displayed to the user is actually represented by an XML defining the UI and a .NET assembly codebehind defining the logic behind the UI. Basically this mechanism is a lot like WPF’s XAML but with several key differences:

  1. DuetML is based on top of WinForms and has very limited UI capabilities specifically designed for Duet’s UI screens
  2. XAML gets compiled (to BAML) during design time while DuetML is interpreted during runtime.
    This gives us a lot of customization & translation capabilities on the server where we process the DuetMLs before sending them to the user (I will probably have to dedicate a whole post specifically to discuss business customization and localization)

In fact, DuetML’s schema was designed to closely resembles XAML to allow transition to XAML in the future…

So, Why Use Metadata?

Simplified solution development. The metadata-based architecture makes it possible to develop declarative business solutions. Because solutions are defined in metadata, developers can build simple, powerful solutions with little or no coding using pre-made common solution building blocks. Advanced developers can use .NET further customize their applications and add new features.

Centralized deployment. In order to run Duet solutions all the clients need is the Duet runtime which is a generic add-in that runs inside the Office applications and can load and interpret the metadata fetched from the server. The different solutions are deployed on a central server and do not require additional installations on the client.

Enhanced customization capabilities. Declarative approach also allows enhanced customization capabilities as customizing the application simply means editing its metadata. Duet’s ACE (Application Configuration Environment) goes a step further by allowing providing a configuration environment that allows the business administrator to apply such changes without requiring knowledge about the metadata and code behind a solution.

Automatic generation and modeling. A declarative approach for defining solution is the basis for enabling modeling and automatic generation of solutions. Imagine that you could quickly select some objects from the backend, define the matching object in Outlook and the data you wish to see and a solution will automatically be generated for you? And imagine you could customize such a solution using a visual programming tool such as Visual Composer (Microsoft Popfly, Yahoo Pipes style of development for you non SAPers)

Tags: , , , , , , ,

Duet Behind the Scenes

Duet, SAP May 4th, 2006

And now, the Duettm architecture as you’ve never seen it before (drawn by an unknown artist):

Duet Architecture

Artistic isn’t it?

On the next few weeks I’ll try to create a series of posts regarding the design and architecture of Duettm .
In the meantime, some quick facts from behind the scenes:

  • The Duettm client is built using the .NET 2.0 framework and an enhanced version of Microsoft Information Bridge Framework (IBF).
  • The first version of Duettm concentrates on scenarios in the Outlook environment.
  • Duettm is a joint effort with Microsoft and SAP and was developed by teams at 6 different worldwide locations.

If you have any technical question or specific areas you would like to hear about in future posts feel free to let me know…

Tags: , , , , , ,