MDX queries and T4 Templates

When I need to display data from a cube on a web page, I usually store the MDX query in a text file and make this file part of a resource file.  This allows me to store the MDX in a readable format.

In the image below, I have two files in the App_GlobalResources folder.  The first is the resource file and the second the text file the contains the query.  One the left you can see that the text file is embedded in the resource file.

The MyQuery.txt file may contain something simple such as

SELECT
    {
        [Measures].[Reseller Sales Amount]
    } ON COLUMNS,
    {
        [Date].[Calendar].[Month].&[<<YEAR>>]&[<<MONTH>>]
    } ON ROWS
FROM
    [Adventure Works]

To use this script in code you can simply access the resource as below.

string mdx = Resources.MDX.MyQuery;

AdomdCommand com = new AdomdCommand(mdx,cxn);

Notice that the Date hierarchy contains the strings <<YEAR>> and <<MONTH>>.  I do this to create place holders so that I can specify different values to adjust the query.

string mdx = Resources.MDX.MyQuery;

mdx = mdx.Replace("<<YEAR>>","2008");
mdx = mdx.Replace("<<MONTH>>","3");

I read an article about T4 templates the other day and wondered if I could replace my text file resources with these templates.  That’s not to say there is anything wrong with the method above.  I have been using this method for quite a while now and haven’t had any problems.  Really I just wanted to try it in order to learn a bit more about T4 templates.

To start with I add a new Preprocessed Text Template to my web project.  I choose to store these templates in a specific folder in App_Code but you don’t have to.

When you add a template (using Visual Studio 2010), the IDE creates two files, one with a .tt extension and another with a .cs extension.  The code in the .cs file is auto-generated and a comment at the top advises against changing this file directly.

To start with the MyQuery.tt file only contains one line, a directive.

The query above contained two placeholders that allowed values to be set a runtime and I want to be able to do the same thing using templates.  There are a couple of ways to do this.  First then with parameters.

<#@ template language="c#" #>
<#@ parameter name="Year" type="System.Int32"#>
<#@ parameter name="Month" type="System.Int32"#>
SELECT
    {
        [Measures].[Reseller Sales Amount]
    } ON COLUMNS,
    {
        [Date].[Calendar].[Month].&[<#=Year#>]&[<#=Month#>]
    } ON ROWS
FROM
    [Adventure Works]

When I save the template file Visual Studio will create a class in the .cs file for me that will render my template.  There is actually quite a lot of code generated and I believe this is done so that you don’t have to distribute the T4 assemblies with the code.

Now to use the template just create an instance of this class, set the parameters and call the TransformText() method.  If you’re not sure of the class name, open the .cs file.

using MDXTemplates;
...
...
    // Create an object from the class
    MyQuery m = new MyQuery();

    // Set the parameters
    m.Session = new Dictionary<string,object>();

    m.Session.Add("Year",2008);
    m.Session.Add("Month",3);

    // Call Initialize to copy the parameters across.
    m.Initialize();

    // Get the MDX

    string mdx = m.TransformText();

Instead of using parameters, you can also add properties and methods to the class (it is marked as partial).  First, go back to the MyQuery.tt file and remove the two parameter directives.

<#@ template language="c#" #>
SELECT
    {
        [Measures].[Reseller Sales Amount]
    } ON COLUMNS,
    {
        [Date].[Calendar].[Month].&[<#=Year#>]&[<#=Month#>]
    } ON ROWS
FROM
    [Adventure Works]

Next create a new class in the same namespace and with the same name as the generated class (don’t forget the partial keyword).

namespace MDXTemplates
{
    public partial class MyQuery
    {
        public int Year{get;set;}
        public int Month{get;set;}
    }
}

The class can now be used like this

using MDXTemplates;
...
...
    // Create an object from the class
    MyQuery m = new MyQuery();

    m.Year = 2008;
    m.Month = 3;

    string mdx = m.TransformText();

Taking this one step further, by adding a constructor to this class….

public partial class MyQuery
{
    public MyQuery(int year,int month)
    {
        Year = year;
        Month = month;
    }
    public int Year{get;set;}
    public int Month{get;set;}
}

I can now load the MDX in one line.

string mdx = new MyQuery(2008,3).TransformText();


AsyncFileUpload file filter

The AsyncFileUpload control (part of the AjaxControlToolkit) does not expose a method that enables you to limit or filter the types of files that can be uploaded.

You could add logic to the server side event UploadedComplete to check the file extension prior to saving the file but you would need to upload the file before it could be checked.

Instead perform the test in the client side event OnClientUploadStarted.  At first I thought that returning ‘false’ from this function would prevent the file uploading, it doesn’t.  You need to raise an error within this function to stop the file upload.  When the error occurs the OnClientUploadError function is called (if you have created a handler for this event).

<head>
    <script type="text/javascript">

        function uploadComplete(sender,args)
        {
            var file = args.get_fileName();
            // Do something here to indicate the file has been uploaded.
        }

        function uploadStarted(sender,args)
        {
            var file = args.get_fileName();

            // File types that should not be uploaded.
            var extensions = "exe|bat|com|msi";

            if (new RegExp("("+extensions+")$").test(file))
            {
                throw(new Error());
            }
        }

        function uploadError(sender,args)
        {
            // Do something to indicate error
        }
    </script>

    <script runat="server">
        void afu_UploadComplete(object sender,AsyncFileUploadEventArgs e)
        {
            // Save File
        }
        void afu_UploadFileError(object sender,AsyncFileUploadEventArgs e)
        {
            // Handle Error
        }
    </script>
</head>
<body>
    <form runat="server>
        <ajx:ToolkitScriptManager runat="server" id="sm1"/>

        <ajx:AsyncFileUpload runat="server" id="afu"
            OnUploadComplete="afu_UploadComplete"
            OnUploadFileError="afu_UploadFileError"
            OnClientUploadStarted="uploadStarted"
            OnClientUploadComplete="uploadComplete"
            OnClientUploadError="uploadError"
            UploaderStyle="Modern"
        />

    </form>
</body>


Database Mirroring

It’s not very often that I need to setup database mirroring and because it’s something I do rarely, I can never remember the exact syntax.  In the code below ServerP is the Principle, ServerM the mirror and ServerW the witness.

-- On ServerP
CREATE ENDPOINT [Mirror] AUTHORIZATION [domain\sqladmin]
    STATE=STARTED
    AS TCP (LISTENER_PORT=7777, LISTENER_IP=ALL)
    FOR DATA_MIRRORING (
        ROLE=PARTNER,
        AUTHENTICATION=WINDOWS NEGOTIATE,
        ENCRYPTION=REQUIRED ALGORITHM RC4
    )
GO
BACKUP DATABASE [DatabaseName] TO DISK 'C:\SomeFolder\DBName.Bak'
GO
BACKUP LOG [DatabaseName] TO DISK 'C:\SomeFolder\DBName.trn'
GO

Copy the .bak and .trn file to ServerM and then execute the following.

-- On ServerM
CREATE ENDPOINT [Mirror] AUTHORIZATION [domain\sqladmin]
    STATE=STARTED
    AS TCP (LISTENER_PORT=7777, LISTENER_IP=ALL)
    FOR DATA_MIRRORING (
        ROLE=PARTNER,
        AUTHENTICATION_WINDOWS NEGOTIATE,
        ENCRYPTION=REQUIRED ALGORITHM RC4
    )
GO
CREATE DATABASE [DatabaseName]
GO
RESTORE DATABASE [DatabaseName]
    FROM DISK='C:\SomeFolder\DBName.Bak'
    WITH NORECOVERY, REPLACE
GO
RESTORE LOG [DatabaseName]
    FROM DISK='C:\SomeFolder\DBName.Bak'
    WITH NORECOVERY, REPLACE
GO
ALTER DATABASE [DatabaseName] SET PARTNER='TCP://ServerP:7777';
GO

On the Witness Server ServerW execute the following.

-- On ServerW
CREATE ENDPOINT [Mirror]
    AUTHORIZATION [domain\sqladmin]
    STATE=STARTED
    AS TCP (LISTENER_PORT=7777,LISTENER_IP=ALL)
    FOR DATA_MIRRORING (
        ROLE=WITNESS,
        AUTHENTICATION=WINDOWS NEGOTIATE,
        ENCRYPTION=REQUIRED ALGORITHM RC4
    )
GO

Finally, back on the principle server ServerP execute these two commands.

-- On ServerP
ALTER DATABASE [DatabaseName] SET PARTNER='TCP://ServerM:7777';
GO
ALTER DATABASE [DatabaseName] SET WITNESS='TCP://ServerW:7777';
GO


Powershell – Processing Production SSAS Databases

In an earlier post I demonstrated a method I use to update development SSAS databases using powershell.  While this is okay for development servers it doesn’t really work with production boxes containing millions of rows of data.

Generally when I refresh a cube, the only data that will have changed relates to the most recent month or couple of months and so it doesn’t really make a lot of sense to perform a full refresh each day on every measure group.

Before you can apply different processing methods to an individual measure group, you need to partition the measure group.  In my scenario it made sense to partition each measure group into calendar quarters and this meant that on a daily basis I could perform a full update on the partitions containing the current and previous quarter data and default processing to the remaining partitions.  I also decided that once a week I would perform a full update to all the partitions.

Read more on creating partitions and slices in measure groups.

Okay, so I have a measure group named Sales.   I have partitioned the data into calendar quarters (by the Sale Date attribute) and added slice information to each partition.  For example the slice for the current quarter looks like [Sale Date].[Quarter Key].&[201104].

The first thing my Powershell script needs to do is determine the names of the two quarters that will be processed fully.

# Create any array to hold the quarter strings
$quarters = "","";

# Determine which quarter we are currently in.
[int] $q = ((get-date).Month+1)/3);

# Create the strings holding the quarter information.
$quarters[0] = "&["+(get-date).Year.ToString()+$q.ToString().PadLeft(2,'0')+"]";

# If current quarter is 1 then we also need Q4 of last year.
if ($q -eq 1)
{
    $quarters[1] = "&["+((get-date).Year-1).ToString()+"04]";
}
else
{
    $quarters[1] = "&["+((get-date).Year).ToString()+
        ($q-1).ToString().PadLeft(2,'0')+"]";
}

If you test this script $quarters[1] and [2] should contain &[201104] and &[201103].  Any partition that ends with either of these strings will have Full processing applied.  I chose to do it this way as I have several measure groups all partitioned into calendar quarters but the slice attribute is different on each.  However all the slices end with the string &[YYYYQQ].

The next part of the script connects to the database and processes the dimensions.

# Connect to SSAS database
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.AnalysisServices");

$server = new-object Microsoft.AnalysisServices.Server;
$server.connect("MY_SSAS_SERVER");

$database = $server.databases["MY_SSAS_DATABASE"];

# Process the dimensions - if you perform a full update on a dimension
# then you have to perform full updates on related measure groups.
# I check the day of the week and only perform a full update on Saturdays

$database.dimensions | foreach-object
{
    if ((get-date).DayOfWeek -eq 'Saturday')
    {
        $_.Process("ProcessFull");
    }
else
    {
        $_.Process("Process Default");
    }
}

Finally process each measure group.  If the day is Saturday then just perform a full update against each partition (all measure groups have at least one partition).  If is not Saturday then look at each partition and if the slice property is not null and ends with either of a values in the $quarters[] array, apply a ProcessFull otherwise ProcessDefault.

# Iterate the cubes
$database.cubes | foreach-object
{
    # Iterate each measure group
    $_.MeasureGroups | foreach-object
    {
        $_.Partitions | foreach-object
        {
            if ((get-date).DayOfWeek -eq 'Saturday')
            {
                $_.Process("ProcessFull");
            }
            else
            {
                if ($_.Slice -ne $Null -and ($_.Slice.EndsWith($quarters[0])
                        -or ($_.Slice.EndsWith($quarters[1])))
                {
                    $_.Process("ProcessFull");
                }
                else
                {
                    $_.Process("ProcessDefault");
                }
            }
        }
    }
}


SQL 2012 – Windowed Functions

I recently installed a copy of SQL2012 RC0 and wanted to look at the new language features that have been added.  In particular the improvements to the OVER clause caught my attention as I could think of a scenario where this would be useful.

Back in March of this year I created a blog entry called Iterative Updates that demonstrated a method of using a common table expression to perform updates to a set of rows, where the updated values where dependent on the values in earlier rows.

In that acticle I used the example of updating mileage lines with a cost.  The following is a snippet from the original post.

On both of the mileage rows I need to update the claim amount column with the total that should be paid. For mileage less that 10K, say it’s 40p per mile and 25p after that. If I processed the rows one at a time I would start the first row, since the user has already claimed 9,975 miles only a further twenty five will be payable at the higher rate and the remainder at the lower rate.

The CTE based query was approximately 40 lines long and I was interested to see if I could achieve the same result using the improved OVER clause and if I could reduce the amount of code required.

After 30 minutes or so of trying different approaches and coming up with wrong answers, I came up with the following query that gave the expected results.

DECLARE
    @PreviousMiles      INT = 9975,
    @ClaimID            INT = 324

UPDATE
    C
SET
    ClaimAmount = CASE
        WHEN C.Mileage+C.PrevMiles<10000 THEN (0.4 * Mileage)
        WHEN C.PrevMiles>=10000 THEN (0.25 * Mileage)
        ELSE (0.4 * (10000-C.PrevMiles))+(0.25 * (C.Mileage+C.PrevMiles-10000))
    END
FROM
    (
        SELECT
            ClaimAmount,
            ClaimLineID,
            Mileage,
            @PreviousMiles+ISNULL(SUM(Mileage) OVER (
                ORDER BY ClaimNo ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING
            ),0) AS PrevMiles
        FROM
            dbo.ClaimLine
        WHERE
            ClaimID = @ClaimID AND Mileage!=0
    ) AS C

So how does this work?  Well for each row containing mileage I need to know how many miles have already been used.  For the first row thats easy, its just the value contained in the @PreviousMiles variable.  For each following row I need the take the @PreviousMiles variable and add any mileage used on previous lines.

At first I started with a query similar to this.

DECLARE @PreviousMiles INT = 9975

SELECT
    @ClaimLineID,
    @Mileage,
    @PreviousMiles,
    @PreviousMiles+SUM(Mileage) OVER (ORDER BY ClaimNo ROWS UNBOUNDED PRECEDING)
FROM
    dbo.ClaimLine
WHERE
    Mileage!=0

This created the result below.  The forth column shows the result of the OVER clause.  You can see that the columns value is made up from the @PreviousMileage amount added to the mileage for each claim line and the previous claim line.  i.e. ClaimLineID 1, is 30+9975 = 10005 whereas ClaimLineID 2, is 30+35+9975.

ROWS UNBOUNDED PRECEDING sets the scope of the SUM() function to allow the rows prior to the current row (and the current row) based on the ORDER BY clause.  In fact the statement is equivalent to ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW.

Not quite right though, as I need the 4th column to indicate the mileage used prior to the current row.  I quickly changed the line to subtract the current mileage.

@PreviousMiles+SUM(Mileage) OVER (ORDER BY ClaimNo ROWS UNBOUNDED PRECEDING)-Mileage

The results were now correct but I didn’t really like the query.  The good news is as well as using UNBOUNDED PRECEDING/FOLLOWING and CURRENT ROW you can also indicate as offset for PRECEDING/FOLLOWING.

@PreviousMiles+ISNULL(SUM(Mileage) OVER (ORDER BY ClaimNo ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING),0)

I have wrapped the SUM() OVER() clause in an ISNULL function as the first row will not have a 1 PRECEDING row and @PreviousMiles+NULL will always be NULL.

From here the rest of the code is pretty straight forward.

In this example I was only updating one claim but the code could be adapted to update multiple claims by using the PARTITION BY clause.

SUM(Mileage) OVER (PARTITION BY ClaimID ORDER BY ClaimNo
    ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING)


Maintaining scroll position on postbacks.

Whenever I create web pages with areas of scrollable content I want to ensure the current scroll position is maintained when a postback occurs (regardless if it is a full or partial postback).

As per the screen shot below, I have created a simple web page containing a div with the style attribute overflow:scroll.  The button just causes a postback to occur (there is no code within the event handler).

To ensure the scroll bars remain in the same position, I use two hidden fields (one for the vertical and another for the horizontal scroll) and a small JQuery script.

The code below uses an update panel to create a partial postback but this method works on full postbacks as well.

Notice that the hidden fields are initialised to zero, this is important as the JQuery script that follows will attempt to set the scroll positions to the values held within these fields on every page load (including the first).

<asp:ScriptManager runat="server" id="sm1"/>

<asp:HiddenField runat="server" id="hfScrollTop" Value="0"/>
<asp:HiddenField runat="server" id="hfScrollLeft" Value="0"/>

<asp:UpdatePanel runat="server" id="upTest">
    <ContentTemplate>

        <div id="myDiv" style="width:300px;height:150px;overflow:scroll;">
            Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
            eiusmod tempor incididunt ut labore et dolore magna aliqua.
            Ut enim ad minim veniam, quis nostrud exercitation ullamco
            ....
        </div>

        <asp:Button runat="server" id="btnPost" Text="Post" OnClick="btnPost_Click"/>
    </ContentTemplate>
</asp:UpdatePanel>

To restore the scroll positions, first I need to save each into the two hidden fields and then when the page (or part of) is reloaded, set the scroll bars position according to the hidden field values.

First then to store the values, I use an anonymous function that captures the scroll event.  This saves the horizontal and vertical scroll positions into the corresponding hidden fields.

$("myDiv").scroll(function()
{
    $("#<%=hfScrollTop.ClientID%>").val($(this).scrollTop());
    $("#<%=hfScrollLeft.ClientID%>").val($(this).scrollLeft());
});

The client identifiers for runat=”server” controls can vary depending on where they are used, .NET ensures the names of controls remain unique by prepending parent id’s. For example my hfScrollTop becomes Contents_CRight_hfScrollTop because I am using nested master pages (One ContentPlaceHolder named ‘Contents’ and another ‘CRight’).  I don’t want to have to examine the source to determine the id everytime I use this script on a new page and so I use <%= Control.ClientID%> to resolve the client id.

To restore the values after a postback simply reverse the assignments above.

$("#myDiv").scrollTop($("#<%=hfScrollTop.ClientID %>").val());
$("#myDiv").scrollLeft($("#<%=hfScrollLeft.ClientID %>").val());

These two scripts need to be contained within the $(document).ready() function (or a sub-function) and in order for the script to work with partial postbacks you need to wrap the whole thing inside the pageLoad() function.  The full script is shown below.

<script type="text/javascript">

function pageLoad()
{
    $(document).ready(function()
    {
        // This function will be called whenever the scroll position
        // (Horizontal or Vertical) changes.
        $("#myDiv").scroll(function()
        {
            $("#<%=hfScrollTop.ClientID %>").val($(this).scrollTop());
            $("#<%=hfScrollLeft.ClientID %>").val($(this).scrollLeft());
        });

        // As the page loads set the scroll bar positions to the
        // values stored in the hidden fields.
        $("#myDiv").scrollTop($("#<%=hfScrollTop.ClientID %>").val());
        $("#myDiv").scrollLeft($("#<%=hfScrollLeft.ClientID %>").val());
    });
}
</script>


Dynamic Forms in ASP.NET Part 3

In the first post in this series I looked at the database tables and procedures, in the second the classes that make up the question controls.  In this final part I will build the Survey class and show how to use it on a web page.

Link to Part 1  Link to Part 2

You will recall from the last post that each of the question classes inherited from QuestionBase which itself inherited from TableRow.  The Survey class will therefore inherit from Table.

There are only three properties for this class PersonID, SurveyID and ValidationGroup.  The first two are needed to retrieve the correct records from the database while the later is used to setup the correct validation group.  In this first code snippet I have only included the boiler plate code.  I will describe the methods CreateControlHierarchy() and Save() below.

namespace DQ.WebControls
{
    public class Survey : Table, INamingContainer
    {
        public int SurveyID{get;set;}
        public int PersonID{get;set;}
        public string ValidationGroup{get;set;}

        protected override void DataBind(){
            Controls.Clear();
            ChildChildControlState();
            CreateControlHierarchy(true);
            ChildControlsCreated=true;
        }
        protected override void CreateChildControls(){
            Controls.Clear();
            CreateControlHierarchy(false);
        }

        // This method creates the various question classes
        // and loads data as appropriate.
        private void CreateControlHierarchy(bool loadData){
            // TODO
        }

        // Save stores the information entered to the database.
        public void Save(){
            // TODO
        }

        // Helper method to convert DBNull.Value to null.
        private object DbNull(object o){
            return (o==DBNull.Value?null:o);
        }

        // OnInit is used to register the control for control
        // control state and the next two methods Load/Save
        // said state.
        protected override void OnInit(EventArgs e){
            Page.RegisterRequiresControlState(this);
            base.OnInit(e);
        }
        protected override void LoadControlState(object savedState){
            object[] state = savedState as object[];
            base.LoadControlState(state[0]);
            SurveyID = (int)state[1];
            PersonID = (int)state[2];
            ValidationGroup = (string)state[3];
        }
        protected override object SaveControlState(){
            object baseState = base.SaveControlState();
            return new object[] { baseState, SurveyID, PersonID, ValidationGroup };
        }
    }
}

CreateControlHierarchy() will call the stored procedure dbo.SELECT_SURVEY (from the first post in this series) and create each question class as required.

private void CreateControlHierarchy(bool loadData)
{
    // Don't process unless parameters have been initialized
    if (PersonID==null || SurveyID==null)
        return; 

    // Create a connection to the database and load the appropriate
    // questions and answers.

    using (SqlConnection sqlConn = new SqlConnection("Data Source Here"))
    using (SqlCommand sqlCom = new SqlCommand("dbo.SELECT_SURVEY",sqlConn))
    {
        sqlConn.Open();

        sqlCom.CommandType = CommandType.StoredProcedure;

        sqlCom.Parameters.Add(new SqlParameter("@SurveyID",SurveyID));

        sqlCom.Parameters.Add(new SqlParameter("@PersonID",PersonID));

        SqlDataReader sqlDR = sqlCom.ExecuteReader();

        // I am not going to show any error handling/checking
        // here 

        // Iterate through the recordset, create and add each
        // question class to the controls collection.

        while (sqlDR.Read())
        {
            // Create a QuestionBase class
            QuestionBase qb;

            // and an object to hold the answer.
            object answer = new object();

            // Use the class field to determine the type of
            // question class to create
            switch (sqlDR["Class"].ToString())
            {
                default:
                case "String":
                    // Create a QuestionString class
                    qb = new QuestionString();
                    // Load the answer from the appropriate db column
                    //
                    // DbNull converts DBNull.Value to null
                    answer = DbNull(sqlDR["VarcharAnswer"]);
                    break;
                case "Date":
                    qb = new QuestionDate();
                    answer = DbNull(sqlDR["DateAnswer"]);
                    break;
                case "Integer":
                    qb = new QuestionInteger();
                    answer = DbNull(sqlDR["IntegerAnswer"]);
                    break;
                case "Double":
                    qb = new QuestionDouble();
                    answer = DbNull(sqlDR["FloatAnswer"]);
                    break;
                case "List":
                    qb = new QuestionList();
                    answer = DbNull(sqlDR["IntegerAnswer"]);
                    break;
                case "Boolean":
                    qb = new QuestionBoolean();
                    answer = DbNull(sqlDR["BooleanAnswer"]);
                    break;
            }

            // Create Unique ID
            qb.ID = sqlDR["QuestionID"].ToString();

            // Load generic parameters
            qb.MaxLength = Convert.ToInt16(sqlDR["MaxLength"]);
            qb.ListID = Convert.ToInt32(sqlDR["ListID"]);
            qb.Mandatory = Convert.ToBoolean(sqlDR["Mandatory"]);
            qb.QuestionText = sqlDR["QuestionText"].ToString();
            qb.ValidationGroup = ValidationGroup;

            // If Databinding load the data.
            if (loadData)
            {
                qb.Answer=answer;
                qb.DataBind();
            }
            Controls.Add(qb);
        }
        sqlConn.Close();
    }
}

Finally we have the Save() method

public void Save()
{
    EnsureChildControls();

    // Create a data connection and initialize a command object.
    using (SqlConnection sqlCon = new SqlConnection("Data Source Here"))
    using (SqlCommand sqlCom = new SqlCommand("dbo.UPDATE_ANSWER",sqlCon))
    {
        sqlCon.Open();
        sqlCommand.CommandType = CommandType.StoredProcedure;

        // Iterate through the child controls (all will be of type
        // QuestionBase), determine the correct database and build
        // a SqlParameter.

        foreach(QuestionBase qb in Controls)
        {
            SqlParameter sqlParam = new SqlParameter();

            sqlParam.Value = (qb.Answer==null || qb.Answer.ToString()==string.Empty
                ?DBNull.Value : questionBase.Answer
            );

            // We have the value of sqlParam now to set the column name
            switch (qb.DataType)
            {
                case UIDataType.Boolean:
                    sqlParam.ParameterName="BooleanAnswer";
                    break;
                case UIDataType.Double:
                    sqlParam.ParameterName="FloatAnswer";
                    break;
                case UIDataType.Date:
                    // Dates are a bit strange so check value again and
                    // convert if needed.
                    sqlParam.ParameterName="DataAnswer";
                    sqlParam.Value=(qb.Answer==null ||
                        qb.Answer.ToString()==string.Empty
                        ?SqlDateTime.Null:Convert.ToDateTime(qb.Answer)
                    );
                    break;
                case UIDataType.Integer:
                    sqlParam.ParameterName = "IntegerAnswer";
                    break;
                case UIDataType.String:
                    sqlParam.ParameterName="VarcharAnswer";
                    break;
            }

            // As we are looping over each record or QuestionClass
            // clear the SqlCommand parameter collection.
            sqlCom.Parameters.Clear();

            // Add parameters
            sqlCom.Parameters.Add(new SqlParameter("@PersonID",PersonID));
            sqlCom.Parameters.Add(new SqlParameter("@SurveyID",SurveyID));
            sqlCom.Parameters.Add(new SqlParameter("@QuestionID",int.Parse(qb.ID)));
            sqlCom.Parameters.Add(sqlParam);

            // Execute the command

            sqlCom.ExecuteNonQuery();
        }
    }
    sqlCon.Close();
}

Now to use the control.  Ensure you have a reference to the namespace containing the Survey control at the top of page or in your web config.

<form runat="server">
    <AjaxToolkit:ToolkitScriptManager runat="server" id="sm1"/>

    <dq:Survey runat="server" id="Survey1" ValidationGroup="vgSurvey"/>

    <asp:Button runat="server" id="btnUpdate" Text="Save" OnClick="btnUpdate_Click" ValidationGroup="vgSurvey"/>
</form>

The code behind will look something like this.

protected void Page_Load(object sender,EventArgs e)
{
    if (!IsPostBack)
    {
        Survey1.PersonID = 1;
        Survey1.SurveyID = 1;
        Survey1.DataBind();
    }
}

protected void btnUpdate_Click(object sender, EventArgs e)
{
    Survey1.Save();
}



Dynamic Forms in ASP.NET Part 2

In a previous post I started looking at a sample application that could dynamically create forms in ASP.NET and then store the user input into a database.

In this post I am going to continue by looking at the classes used to generate the web controls.

As I mentioned in the first post I took the basic ideas from this article on MSDN by Scott Mitchell (2004).  My first few attempts mirrored this code almost exactly and there is still quite a bit of code in my solution that came from reading this.  Just how much, I’ll leave to you to decide.

One big change I made was to move away from using UserControls as I came across problems with partial page rendering and had to rely on nasty hacks to get the forms to work properly.  In the above article each question is a user control that implements an interface.  Initially I kept the idea of the interface and created a class for each question.  It soon became apparent that I had lots of duplicated code and in the early stages of development a simple change would require me to change each Question class.  For this reason I moved away from an Interface and created an abstract base class instead.

I decided that my Survey class would inherit from Table, and each Question from TableRow (This just helps to keep the layout neat).  The class diagram is displayed below.

The QuestionBase class has two abstract properties and one astract method that classes inheriting must implement. DataType – which is an enumeration, this is used to determine which column in the Answer table to read / update.  PostBackControl – The base class handles the LoadPostData event and in the default implementation it also creates a required field validator (if the mandatory flag for a question a set).  In order to do both of these things the class need to know which control to attach to.  The method is CreateAnswerCell() – Each class is responsible for rendering the controls it needs to create the question control and required validators (other than the required validator previously mentioned).

You can also override the method CreateQuestionCell() and CreateValidatorCell() if you don’t want the base implementation or you can override CreateControlHierarchy() if you want the child classes to control the rendering of the entire TableRow.

Okay, it will probably make more sense with the code, so here is the implementation of QuestionString.

namespace DQ.WebControls
{
    public class QuestionString : QuestionBase
    {
        private TextBox textBox;

        public override UIDataType DataType{get{return UIDataType.String;}}

        protected override Control PostBackControl{get{return textBox;}}

        protected override void CreateAnswerCell(bool loadData)
        {
            TableCell tcAnswer = new TableCell();

            textBox = new TextBox();
            textBox.ID = "textbox";
            textBox.MaxLength=MaxLength;

            tcAnswer.Controls.Add(textBox);

            if (loadData)
                textBox.Text=(Answer==null?string.Empty:Answer.ToString());

           Controls.Add(tcAnswer);
        }
    }
}

Seems pretty simple really.  All the real work is done in QuestionBase.

namespace DQ.WebControls
{
    public abstract class QuestionBase : TableRow,
            INamingContainer, IPostBackDataHander
    {
        // Most questions will have the option of being mandatory and so
        // it makes sense to place the validator in the base class.
        protected RequiredFieldValidator rfv;

        // Properties
        public string QuestionText{get;set;}
        public string ValidationGroup{get;set;}
        public int MaxLength{get;set;}
        public int ListID{get;set;}
        public bool Mandatory{get;set;}
        public object Answer{get;set;}

        // Each question is strongly typed.  Each question class
        // must provide this information.
        public abstract UIDataType DataType{get;}

        // Each child class should override this property and
        // return the control that will accept input.
        protected abstract Control PostBackControl{get;}

        // The databind method is called when populating
        // the controls with info from the database.
        public override void DataBind()
        {
            base.DataBind();
            Controls.Clear();
            ClearChildControlState();
            CreateControlHierarchy(true);
            ChildControlsCreated=true;
        }

        // This method recreates controls on postback
        protected override void CreateChildControls()
        {
            Controls.Clear();
            CreateControlHierarchy();
        }

        // You can override this method if you want
        // to control the TableRow layout
        protected virtual void CreateControlHierarchy(bool loadData)
        {
            CreateQuestionCell(loadData);
            CreateAnswerCell(loadData);
            CreateValidatorCell(loadData);
        }

        // The question cell is simple.  Just text
        // in a table cell.
        protected virtual void CreateQuestionCell(loadData)
        {
            TableCell tcQuestion = new TableCell();
            tcQuestion.Text = QuestionText;
            Controls.Add(tcQuestion);
        }

        // The answer cell is abstract, child class must
        // provide this implementation.
        protected abstract void CreateAnswerCell(bool loadData);

        // The validator cell is straight forward enough.  It
        // uses the property PostBackControl to attach to the
        // input control.
        protected virtual CreateValidatorCell(bool loadData){
            TableCell tcValidator = new TableCell();

            if (Mandatory){
                rfv = new RequiredFieldValidator();
                rfv.ID = "validator";
                rfv.ControlToValidate = PostBackControl.ID;
                rfv.Display = ValidatorDisplay.Dynamic;
                rfv.ErrorMessage="*";
                rfv.CssClass="ValidationError";
                rfv.ValidationGroup=ValidationGroup;

                tcValidator.Controls.Add(rfv);
            }

            Controls.Add(tcValidator);
        }

        // Register the control for both ControlState and PostBackEvents
        protected override void OnInit(EventArgs e)
        {
            Page.RegisterRequiresPostBack(this);
            Page.RegisterRequiresControlState(this);
            base.OnInit(e);
        }
        // I don't use this method but it has to be implemented.
        void IPostBackDataHandler.RaisePostBackDataChangedEvent(){}

        // LoadPostData retrieves the value entered and stores in the
        // Answer property.
        // The PostBackControl property is used as the key to retrieve
        // the correct value from the collection.
        bool IPostBackDataHandler.LoadPostData(string postKey,NameValueCollection values)
        {
            EnsureChildControls();
            Answer = values[PostBackControl.UniqueID];
            return false;
        }

        // Finally two methods for control state
        protected override void LoadControlState(object savedState)
        {
            object[] state = savedState as object[];
            base.LoadControlState(state[0]);
            Answer = state[1];
        }
        protected override object SaveControlState()
        {
            object baseState = base.SaveControlState();
            return new object[]{baseState,Answer};
        }
    }
}
public enum UIDataType
{
    String,
    Integer,
    Double,
    Date,
    Boolean
}

Each of the other question classes looks very similar to the QuestionString class and I won’t show the code for each.  I will show two more which implement slightly different behaviour; QuestionDate & QuestionList. Note – QuestionDate uses the AjaxControlToolkit controls CalendarExtender and VallidatorCalloutExtender.

public class QuestionDate : QuestionBase
{
    private TextBox textBox;
    private CalendarExtender ce;
    private CompareValidator cv;
    private ValidatorCalloutExtender vce;

    public override UIDataType DataType{get{return UIDataType.Date;}}

    protected override Control PostBackControl{get{return textBox;}}

    protected override CreateAnswerCell(bool loadData){
        TableCell tcAnswer = new TableCell();

        textBox = new TextBox();
        textBox.ID = "textbox";
        textBox.MaxLength=10;
        textBox.Columns=10;

        tcAnswer.Controls.Add(textBox);

        if (loadData){
            textBox.Text=string.Format("{0:dd/MM/yyyy}",Answer);
        }

        // Add a Calendar Extender
        ce = new CalendarExtender();
        ce.ID = "calendar";
        ce.TargetControlID = textBox.ID;
        ce.Format="dd/MM/yyyy";
        ce.PopupPosition = CalendarPosition.TopLeft;

        tcAnswer.Controls.Add(ce);

        // Add a compare validators to enusre input is a valid
        // date.

        cv = new CompareValidator();
        cv.ID = "compare";
        cv.ControlToValidate = textBox.ID;
        cv.Display = ValidatorDisplay.None;
        cv.ErrorMessage = "Invalid date format.  Use dd/MM/yyyy.";
        cv.Operator = ValidationCompareOperator.DataTypeCheck;
        cv.Type = ValidationDataType.Date;
        cv.ValidationGroup = ValidationGroup;

        tcAnswer.Controls.Add(cv);

        // Finally an an extender for the validator.

        vce = new ValidatorCalloutExtender();
        vce.ID = "extender";
        vce.TargetControlID = compareValidator.ID;
        vce.PopupPosition = ValidatorCalloutPosition.Right;

        tcAnswer.Controls.Add(vce);

        Controls.Add(tcAnswer);
    }
}

QuestionList uses a DropDownList control instead of a textbox.  It also needs to retrieve the list from the database and override the CreateValidatorCell() method as below.

public class QuestionList : QuestionBase
{
    private DropDownList dropDownList;

    public override UIDataType DataType{get{return UIDataType.Integer;}}

    protected override Control PostBackControl{get{return dropDownList;}}

    protected override CreateAnswerCell(bool loadData)
    {
        TableCell tcAnswer = new TableCell();

        dropDownList = new DropDownList();
        dropDownList.ID = "dropdown";
        dropDownList.Width=Unit.Pixel(180);

        tcAnswer.Controls.Add(dropDownList);

        // Retrieve list from database.
        using (SqlConnection sqlConn = new SqlConnection("Data Source Here"))
        using (SqlDataAdapter sqlAdp = new SqlDataAdapter("dbo.SELECT_LIST",sqlConn))
        using (DataTable dt = new DataTable())
        {
            sqlAdp.SelectCommand.CommandType=CommandType.StoredProcedure;

            sqlAdp.SelectCommand.Parameters.Add(
                new SqlParameter("@ListID",ListID)
            );

            sqlAdp.Fill(dt);

            // Add a default row to handler NULL values i.e.
            // where the user is yet to make a selection.
            DataRow dr = dt.NewRow();
            dr[0]="0";
            dr[1]="Please Select";
            dt.Rows.Add(dr);

            dropDownList.DataSource = dt;
            dropDownList.DataValueField=dt.Columns[0].ColumnName;
            dropDownList.DataTextField=dt.Columns[1].ColumnName;
            dropDownList.DataBind();
        }

        if (loadData)
        {
            // Select appropriate listitem
            dropDownList.ClearSelection();

            string answer = (Answer==null?"0":Answer.ToString());

            for (int i=0;i<dropDownList.Items.Count;i++)
            {
                if (dropDownList.Items[i].Value==answer)
                {
                    dropDownList.Items[i].Selected=true;
                    break;
                }
            }
       }
       Controls.Add(tcAnswer);
    }

    // Override CreateValidatorCell as we need to add the property
    // InitialValue to the required field validator.
    protected override void CreateValidatorCell(bool loadData)
    {
        // Create the default implementation.
        base.CreateValidatorCell(loadData);

        // if Mandatory is true then the validator exists
        if (Mandatory)
        {
            // Add the property value
            rfv.InitialValue="0";
        }
    }
}

In the next (final) post on this subject I will create the Survey Web Control and show how to use the control on a web page.



Hyper V & Wireless Connections

I have a Hyper V server at home for testing different scenarios (well its not really a server just a bog standard desktop pc).  On this computer I use a wireless card to connect to my router as the pc is no where near any telephone sockets.  At first I thought I couldn’t connect the virtual computers to the internet (via wireless) and so used a couple of X10 plugs to route the internet through the powerlines.

However, with a bit of tweaking you can use a wireless connection with the virtual computers.  To start with wireless lan is not automatically available on Server 2008 r2, you have to enable the feature.

Open the Server Manager, click on features and then Add Features.  In the dialog scroll down to Wireless LAN Service and check the option.  Click next followed by install and in a few seconds the wireless service will run running.

Next, open the Hyper-V Manager and select Virtual Network Manager from the actions pane.  Create a new internal network, I just left all the settings as they were.

If you already have virtual computers created you can change the network of each by editing the settings of each.  From the settings dialog, select Network Adapter and then select the newly created internal network from the list.

To complete this task, open Network and Sharing Centre by right clicking the tray icon (or via control panel).  Select Change Adapter Settings from the list on the left and then select the Wireless and Internal network items at the same time, right click and select Bridge Connections.

After a few seconds a new icon will appear in the Network Connections window, similar to the screen shot below.  The virtual computers can now use the wireless connection.



Dynamic Forms in ASP.NET

As part of an ASP.NET application I recently created I wanted to add a capability that allowed power users to define set of questions that could be used by the application.  In other words, when the application needed changing to allow additional information to be stored, the users can make the changes themselves using a few administration pages.

For example if I have the following information stored in the database.

I would like to render this dynamically to create the following form.

Okay, it doesn’t look great without any css but you get the idea.

In this post (and the next few) I will walk through a fictitious survey application I created as a proof of concept.

Initially I had the following goals in mind.

  1. Everything should be encapsulated into a server/user control.  I did not want lots of complicated code of each web page that would include these dynamic questions.
  2. The users should be able to specify the data type of each question and the controls should ensure entries are valid.
  3. As well as basic types such a String, Integer etc. I also wanted the forms to be capable of containing lists.
  4. The controls would have to work with partial page postbacks (i.e. ajax / UpdatePanels).

Hopefully I will cover each of these points as I go but for now I am going to concentrate on the database tables and procedures required.

I have a table for People (Person) and another for the Surveys.  Each Survey contains one of more questions and each of these questions can be of a certain type (QuestionType).  At the moment I have six question types String, Integer, Double, Boolean, Date and List.

The PersonSurvey table determines which surveys each person has completed and links to the Answer table.

Finally I have two more tables List and ListItem; these are used exclusively by the List QuestionType.

This schema design is not entirely my own work (in fact most of it is borrowed), take a look at this MSDN post by Scott Mitchell.  I used the ideas in this article as a base for my implementation.

The schema is fairly straight forward but here are just a few points to aid in understanding.

  1. I have created a unique index on the PersonSurveyID column of the PersonSurvey table to allow a foreign key to be created from the Answer table.
  2. The Answer table contains five nullable columns to hold the question answer however only one of these columns will be populated for each row.  I have done this so that the data can be stored in the correct type. As I am using SQL 2008 R2, I have marked each of these columns as SPARSE.  This means SQL will not use any space to store null values.
  3. Question has a foreign key relation to List but not every question will require a list.  For this reason the Question.ListID column is nullable.

There are only three stored procedures needed for the app.  One to Select (read) the questions/answers for a given survey/person, another to Insert or Update the answer and one more to retrieve a list (if required).

CREATE PROC dbo.SELECT_SURVEY
    @SurveyID        INT,
    @PersonID        INT
AS
SET NOCOUNT ON

SELECT
    Q.QuestionText, Q.QuestionID, T.Class,
    Q.MaxLength, Q.Mandatory, ISNULL(Q.ListID,0) AS ListID,
    A.IntegerAnswer, A.FloatAnswer, A.VarcharAnswer,
    A.DateAnswer, A.BooleanAnswer
FROM
    dbo.Survey AS S
JOIN
    dbo.Question AS Q
        ON S.SurveyID            = Q.SurveyID
JOIN
    dbo.QuestionType AS T
        ON Q.QuestionTypeID      = T.QuestionTypeID
LEFT JOIN
    dbo.Answer AS A
        ON Q.QuestionID          = A.QuestionID
LEFT JOIN
    dbo.PersonSurvey AS PS
        ON A.PersonSurveyID      = PS.PersonSurveyID
        AND S.SurveyID           = PS.SurveyID
        AND PS.PersonID          = @PersonID
WHERE
    S.SurveyID                   = @SurveyID
GO

If the person already has completed the given survey then the result entered will be returned otherwise the Answer columns will all be NULL.

To store the answers I will use one procedure.  This procedure will need to create a record in PersonSurvey (if it does not already exist) and then either update or insert a new row into the Answer table.

CREATE PROC dbo.UPDATE_ANSWER
    @SurveyID            INT,
    @PersonID            INT,
    @QuestionID          INT,
    @IntegerAnswer       INT          = NULL,
    @FloatAnswer         FLOAT        = NULL,
    @VarcharAnswer       VARCHAR(255) = NULL,
    @DateAnswer          DATE         = NULL,
    @BooleanAnswer       BIT          = NULL
AS
SET NOCOUNT ON

DECLARE
    @PersonSurveyID      INT          = 0

-- If a record already exists in PersonSurvey, get the identity value
SELECT
    @PersonSurveyID      = PersonSurveyID
FROM
    dbo.PersonSurvey
WHERE
    PersonID             = @PersonID
AND SurveyID             = @SurveyID 

-- If @PersonSurveyID is zero then the record did not exist
IF @PersonSurveyID = 0
    BEGIN
        INSERT INTO dbo.PersonSurvey
        (
            PersonID,
            SurveyID
        ) VALUES (
            @PersonID,
            @SurveyID
        )

        SET @PersonSurveyID = SCOPE_IDENTITY()
    END

-- Insert or Update dbo.Answer?
IF EXISTS (SELECT 1 FROM dbo.Answer WHERE QuestionID = @QuestionID AND PersonSurveyID = @PersonSurveyID)
    BEGIN
        UPDATE
            dbo.Answer
        SET
            IntegerAnswer         = @IntegerAnswer,
            FloatAnswer           = @FloatAnswer,
            VarcharAnswer         = @VarcharAnswer,
            DateAnswer            = @DateAnswer,
            BooleanAnswer         = @BooleanAnswer
        WHERE
            QuestionID            = @QuestionID
        AND PersonSurveyID        = @PersonSurveyID
    END
ELSE
    BEGIN
        INSERT INTO dbo.Answer (
            QuestionID,
            PersonSurveyID,
            IntegerAnswer,
            FloatAnswer,
            VarcharAnswer,
            DateAnswer,
            BooleanAnswer
        ) VALUES (
            @QuestionID,
            @PersonSurveyID,
            @IntegerAnswer,
            @FloatAnswer,
            @VarcharAnswer,
            @DateAnswer,
            @BooleanAnswer
        )
    END
GO

As each Answer column defaults to null, I only have to pass the correct type when calling the procedure.  i.e.

EXEC dbo.UPDATE_ANSWER @SurveyID=1, @PersonID=2, @QuestionID=3, @VarcharAnswer='Test';

One last procedure for returning lists.

CREATE PROC dbo.SELECT_LIST
    @ListID            INT
AS
SET NOCOUNT ON 

SELECT
    ListItemID,
    ListItemName
FROM
    dbo.List
WHERE
    ListID = @ListID
GO

That is all the SQL code needed.  In my next post I will talk through the C# classes that make the question web controls.