SQLite and Fluent NHibernate with “Code First” in C#: Rewriting the OpenCollective Wiki

Introduction
Seven years ago, I wrote a wiki for managing requirements for multiple software development projects. I used C# in a simplistic ASP.NET web forms application with SQL Server as my data store. I published it under the name OpenCollective on Code Project and since then it has been viewed on just over 196,000 occasions and downloaded a little more than 1,600 times.

I still get an occasional question about OpenCollective, so a month or two ago I decided to rewrite it in order to explore using SQLite and Fluent NHibernate in a “code first” approach set in an ASP.NET MVC 3 application written in C#. The original OpenCollective was in some ways an experiment with technologies and techniques I was interested in at that time, so a rewrite to learn and explore something new seemed like a good idea.

The plan is simple. As time allows, I’ll rewrite OpenCollective and blog about my progress whenever it seems I have something of value to share. With some of these posts, I’ll share the entire code base for download. Others may just illustrate something I’ve learned or something I want to preserve for my own future reference. I tackled the first chore over a month ago but work and other activities pushed this project aside for a while, so I’m just getting around to blogging about the code now. I suspect this will be a priority driven pattern for this project.

Code First with Fluent NHibernate and SQLite
The first thing I wanted to accomplish in the OpenCollective rewrite was a move to SQLite and Fluent NHibernate in a “code first” approach to replace the SQL Server schema and the old ADO.NET code in the original. Some quick searching led me to the Fluent NHibernate Getting Started page. (If you haven’t read it, do so now.) Combining that with what I have already learned in exploring SQLite in this blog, I soon had a code first database being generated.

I’ll attach the entire code in 7zip (saves me 25% in bandwidth over standard zip), so I’m not going to go through all of the code in text here, but I do want to share with you some of the key steps in the my code first with Fluent NHibernate and SQLite adventure.

First, create your POCO entity class:

public class Project
{
	public virtual long Id { get; private set; }
	public virtual string Name { get; set; }
	public virtual DateTime Updated { get; set; }

	public virtual Author Author { get; set; }
	public virtual IList<Topic> Topics { get; set; }
	public virtual IList<Attachment> Attachments { get; set; }

	public Project()
	{
		Topics = new List<Topic>();
		Attachments = new List<Attachment>();
	}

	public virtual void AddTopic(Topic topic)
	{
		topic.Project = this;
		Topics.Add(topic);
	}

	public virtual void AddAttachment(Attachment attachment)
	{
		attachment.Project = this;
		Attachments.Add(attachment);
	}
}

Second, create a Fluent NHibernate mapping class:

public class ProjectMap : ClassMap<Project>
{
	public ProjectMap()
	{
		Id(x => x.Id);
		Map(x => x.Name);
		Map(x => x.Updated);
		References(x => x.Author);

		HasMany(x => x.Attachments)
			.Cascade.All();

		HasMany(x => x.Topics)
			.Cascade.All();
	}
}

Third, create a factory class that will return the NHibernate ISessionFactory and generate the SQLite schema and database file:

internal class DataFactory
{
	string _dbFile = string.Empty;
	bool _overwriteExisting = false;

	public DataFactory(string dbFile, bool overwriteExisting)
	{
		_dbFile = dbFile;
		_overwriteExisting = overwriteExisting;
	}

	public ISessionFactory CreateSessionFactory()
	{
		return Fluently.Configure()
			.Database(
				SQLiteConfiguration.Standard
					.UsingFile(_dbFile)
			)
			.Mappings(m => m.FluentMappings.AddFromAssemblyOf<DataFactory>())
			.ExposeConfiguration(BuildSchema)
			.BuildSessionFactory();
	}

	private void BuildSchema(Configuration config)
	{
		if (_overwriteExisting)
		{
			if (File.Exists(_dbFile)) File.Delete(_dbFile);

			var se = new SchemaExport(config);
			se.Create(false, true);
		}
	}
}

Fourth, create a repository class that will let you create the database and perform CRUD operations using your entity classes:

public static class OcDataRepository
{
	public static void Create(string dbFile)
	{
		DataFactory df = new DataFactory(dbFile, true);
		using (var sf = df.CreateSessionFactory())
		{
			sf.Close();
		}
	}

	public static void AddProject(string dbFile, Project project)
	{
		DataFactory df = new DataFactory(dbFile, false);
		using (var sf = df.CreateSessionFactory())
		using (var session = sf.OpenSession())
		using (var trans = session.BeginTransaction())
		{
			session.SaveOrUpdate(project.Author);
			session.SaveOrUpdate(project);
			trans.Commit();
		}
	}
}

Finally, write a little test console app to see if it all works:

class Program
{
	static void Main(string[] args)
	{
		string dbFile = @"C:\temp\test4.db";
		OcDataRepository.Create(dbFile);

		Project project = new Project
		{
			Name = "My First Project",
			Updated = DateTime.Now,
			Author = new Author { Name = "John Smith", Username = "smithj", Email = "jsmith@gmial.com" }
		};
		OcDataRepository.AddProject(dbFile, project);

		Console.WriteLine("done");
		Console.ReadLine();
	}
}

Examine the SQLite db file with your favorite SQLite tool and see the SQL:

CREATE TABLE "Project" (
	Id  integer, 
	Name TEXT, 
	Updated DATETIME, 
	Author_id INTEGER, 
	primary key (Id)
)

If you don’t have 7-zip, download and install it first. Download the code OcWiki.7z (695.8KB)

Estimating Software Features and Fixed Feature Compensation from 8th Light

I just finishing reading a fascinating blog post by Paul Pagel of 8th Light called Fixed Feature. I recommend that you read the post and download and study the linked PDF entitled From Estimate to Commitment.

Companies willing to pay for software development on a time-and-materials basis are fewer and farther between. They don’t have the resources to monitor such work to assure themselves that the team is progressing as it should. Understandably, they cannot shoulder this risk. So they want their teams, internal and external, to give them a fixed bid for a project: fixed time, fixed cost, fixed features.

Pagel points out, “software is a craft with a close problem finding to problem solving loop.” In other words, we’re not always very good at anticipating problems in a development project until they are just about to happen. Sometimes we don’t even see them coming. We can minimize these events through continuous review and refinement of our processes and practices, study and evaluation of past performance versus estimates and the assumptions that went into those. Too often we do not take the time to do so, dooming ourselves to historical repetition.

Rather than attempting to created a fixed bid for an entire project, 8th Light, according to Pagel, estimates each feature using PERT and creates a price point for each feature. At the end of an iteration, they bill the client for features accepted by the client. This is an interesting approach to combining the desire for fixed bid pricing and Agile development.

Of course, this or any variation on fixed bid pricing requires that we improve our track record on creating estimates. If you visit our friends in Mountain View and search for estimating software development, you will find over four million results. There are tools and whitepapers and blog posts galore. But before you spend more money on tools or consultants, look to your own experience. Be willing to record what you’ve done, evaluate it honestly and boldly, and make adjustments based on what you have learned. Surely this will be less painful than blindly repeating the past.

Read Pagel’s work on this subject and let me know what you think.

Attracting and Keeping Top Talent in Software Development Teams

A friend recently shared a Forbes article with me entitled Top Ten Reasons Why Large Companies Fail to Keep Their Best Talent which I found informative and useful in understanding my own recent thoughts on this topic. Rather than review each of those reasons here, I encourage you to read the original. Instead I would like to share with you the flip side of that coin based on my own recent experiences.

Attracting Top Talent
Before you can work on keeping your top talent, you have to find them and convince them to join your team. Here are five things you should be doing to attract top talent:

  1. Choose a competent and effective recruiter. This can make all the difference. Don’t just hire an agency and let them blast a job description to every Tom, Dick and Mary of the tech world. Know specifically who is representing your company. Make sure that she knows how to find and filter top talent for you. Ensure that she has the communication and people skills required to manage the phone and in-person interviews and coordinate with the hiring manager to make his job even easier rather than just dumping a pile of resumes on a desk and waiting to get paid.
  2. Do more than one phone screen. Give at least two top team members the chance to phone screen the candidate. Make sure they are prepared and understand what they should ask. Then have one or two managers or potential peers conduct a phone screen as well. Never rely on the resume alone to decide whether you will do a face to face interview.
  3. Have the candidate interview with more than just the hiring manager. Have potential peers and even potential subordinates interview the candidate as well. And when possible, have a peer or supervisor of the hiring manager conduct an interview. Meet with and collect the thoughts and opinions of every interviewer and carefully consider their input.
  4. Assure that every interviewer is positive and upbeat about your company but equally honest and transparent about the challenges and opportunities within the company which the candidate may be able to help resolve or improve. Don’t paint a dismal picture but don’t put a shine on a dull spot. Any intelligent candidate who gets different stories from interviewers will think twice before accepting an offer. Transparency and honesty from the bottom to the top of the company will be a refreshing and attractive quality. And don’t worry about scaring off a candidate who is afraid of a blemish. You don’t want to hire someone who wants to work for a perfect company with no opportunities to contribute to the solutions of the real problems that every company has.
  5. Follow up and answer a candidates questions after the interview process is complete and make a decision as quickly as possible. If you will have some delays before you can make a decision, keep the lines of communication open and keep the candidate up to date. This kind of follow through is often overlooked and companies often take for granted that the candidate will sit still and wait. They won’t. To keep a fish on a hook, as they say, you have to work the line. Let it go slack, and you’ll lose.

Keeping Top Talent
Once you have hired a key employee, make an opposites list from the Forbes article and work toward eliminating the reasons that top talent walks out the door. Of the ten, here are my top picks recast as things you should do to keep top talent on your team:

  1. Give your employees an opportunity to have a voice in key policy and process decisions. Listen to your people with an open mind, prepared to change your mind if you have overlooked something. Your top talent will often have better ideas than you may think.
  2. Take the time to provide regular feedback to your employees. Annual reviews are great, but follow up with periodic reviews of goals, professional development, projects and opportunities for improvement. And always find a way to share positive feedback. When you acknowledge achievements and performance, publicly and privately, you’ll get even more of the same.
  3. Make sure that your team members know that you care about their professional career development. The small amount of time you invest in helping your team map their future success will yield returns in happy, dedicated employees much greater even than the recent gains in the gold market.
  4. Steer a steady ship that can make tactical adjustments in course but is on a solid strategic heading. If you run into stormy waters, keep faith with your employees and stay on course. If you need to make strategic course changes, involve your top talent in that decision making. Getting on a boat headed for Hawaii and finding out that the captain has decided to go to Alaska instead without even talking to you can be more than demoralizing.
  5. Build teams of people who can work well together, who are talented and skilled and willing to pull their own weight and then some. Passionate people will help to bring out the best in team members who are having a bad day, but it is impossible to fix a team member who fundamentally lacks the requisite skills and desire to acquire them.
  6. Be open minded and tolerant of opposing points of view. If you do not invite honest discussion with your team at appropriate times, you will lose your top talent and end up with a team that affirms any decision you make, even those that will send you off a cliff that you did not have the foresight to recognize.
  7. You can teach management skills to a leader. But teaching leadership skills to a manager is not so easy. Look for leaders who can motivate and rally teams. You can hire a clerk or accountant to take care of the bean counting. But you may not recover from having a leaderless team and the resultant chaos and confusion and serial loss of top talent that will result. Do not be afraid to amputate and stop the bleeding. Keeping a failed manager long beyond the point of recognizing the problem to avoid the pain of change is an ominous sign to your top talent that you lack the leadership required to steer the ship successfully to port and they will abandon ship at the first reasonable moment.

If you’re making New Year’s resolutions with respect to your company, I urge you to review these lists, and the plethora of others available on the web from sources far more authoritative than me. Take positive action to attract and keep your top talent. And if you find yourself looking for a company that exhibits these desirable qualities, keep up your search. They are out there. And while no company is perfect, there are certainly some that far and away exceed others. So whatever you do, don’t give up hope of a better day.

Christmas Gifts That Count

On rare occasions I step away from my primary theme in this blog. Christmas is certainly one of those. Many people I work with and many readers of this blog are not Christian, but I have not yet met or worked with anyone who does not appreciate the Christmas season as a time to exchange gifts with loved ones or enjoy the company of friends and family around a cup of cheer. So to all my friends, coworkers and blog readers, I wish you a very Merry Christmas and hope that this season will find you giving and receiving gifts that count.

What are the gifts that count? To answer that, I want to share with you a selection from a devotional talk given by President Dieter F. Uchtdorf, Second Counselor in the First Presidency of the Church of Jesus Christ of Latter-day Saints (of which I am a member) at the 2011 First Presidency Christmas Devotional.

“Christmas and some of the cherished traditions of the season remind us that we, like the Wise Men of old, should seek the Christ and lay before Him the most precious of gifts: a broken heart and a contrite spirit. We should offer Him our love. We should give Him our willingness to take upon ourselves His name and walk in the path of discipleship. We should promise to remember Him always, to emulate His example, and to go about doing good.

We cannot offer Him the gift of perfection in all things because this is a gift beyond our capacity to give—at least for now. The Lord does not expect that we commit to move mountains. But He does require that we bring as gifts our best efforts to move ourselves, one foot in front of the other, walking in the ways He has prepared and taught.

And what are the Savior’s gifts to those who are willing to bring these gifts to Him?

This may be the most one-sided gift exchange in the history of the universe. The Savior’s gifts to us are breathtaking.

Let us begin with immortality. Because the Savior overcame death, all men and women—both the just and the unjust—will live forever.

Then, forgiveness—even though our sins and imperfections be as scarlet, they can become white as snow because of Him.

And finally, eternal life—the greatest gift of all. Because of the Atonement of Christ, not only are we guaranteed an infinite quantity of life, but He offers the possibility of an unimaginable quality of life as well.

Some of His divine gifts are reserved for that glorious future day when we return to His presence.

But He extends many gifts and His grace to us every day. He promises to be with us, to come to us when we need comfort, to lift us when we stumble, to carry us if needed, to mourn and rejoice with us. Every day He offers to take us by the hand and help transform ordinary life into extraordinary spiritual experiences.”

Whether you celebrate the Christ and His birth this season or not, I wish you all peace and joy and a happy and prosperous new year.

Custom Recursive Model Validation in .NET Using Data Annotations

Need to write your own model validator outside the scope of an application framework such as ASP.NET MVC? A short while ago, I needed to do just that. I was writing a WCF service with a relatively complex data model which required a much greater level of validation than the DataMember attribute’s IsRequired property could provide.

Here’s the solution I found. But first a bit of background.

Validation in ASP.NET MVC 
I’ve been using Data Annotations attributes for view model validation for years in the context of ASP.NET MVC for model validations, client and server side. I always took the server side validation for granted and looked at the client side with greater interest. When getting started with ASP.NET MVC, I used Steve Sanderson’s xVal library. Then I switched to ASP.NET MVC 3’s client side validation.

For client side validation, I’m really starting to like jQuery’s unobtrusive validation and the ASP.NET MVC’s HtmlHelper class and its GetUnobtrusiveValidationAttributes method. And for the server side, view model binding and the ModelState.IsValid property works fine.

Validation Without an App Framework 
But this post is not about client side validation or server side validation in ASP.NET MVC. I had to do the validation against the model in the WCF service which, as far as I know, does not have a nice model validation facility other than its own data model serialization which just throws a nasty exception should the data being passed over the wire not provide required data.

So why not use the same approach used by our friends in the ASP.NET MVC world. A little Google investigation turned up something that looked like exactly what I wanted. I found an answer posted to StackOverflow by Mike Reust which included a link to his DataAnnotationsValidatorRecursive library. (Gotta love StackOverflow.)

After some experimentation, I had made some tweaks to Mike’s code and came up with something that worked the way I wanted it to. Here’s a partial example of a data model and how the ValidateObject with it’s optional MinOccursOnEnumerable can be used to require one or more items to be included in a required complex enumerable type that will get recursively.

[DataMember(IsRequired = true), 
	Required(ErrorMessage = "Employer cannot be null."), 
	ValidateObject]
public Employer Employer { get; set; }


[DataMember(IsRequired = true), 
	Required(ErrorMessage = "Children cannot be null."), 
	ValidateObject(MinOccursOnEnumerable = 1)]
public Child[] Children { get; set; }

Using the recursive validation library is easy. Here’s an example:

//perform validation using DataAnnotations for custom validation messages
List<ValidationResult> results = new List<ValidationResult>();
bool isValid = DataAnnotationsValidator.TryValidateObjectRecursive<MyModelData>(model, results);

//now examine isValid and the results

And here’s the code for the validation extensions and ValidateObject attribute.

using System.Collections;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System;

namespace MyValidator
{
	public static class DataAnnotationsValidator
	{
		public static bool TryValidateObject(object obj, ICollection<ValidationResult> results)
		{
			return Validator.TryValidateObject(obj, new ValidationContext(obj, null, null), results, true);
		}

		public static bool TryValidateObjectRecursive<T>(T obj, List<ValidationResult> results)
		{
			bool result = TryValidateObject(obj, results);

			var properties = obj.GetType().GetProperties().Where(prop => Attribute.IsDefined(prop, typeof(ValidateObjectAttribute)));

			foreach (var property in properties)
			{
				var valAttrib = property.GetCustomAttributes(typeof(ValidateObjectAttribute), true).FirstOrDefault() as ValidateObjectAttribute;
				var value = obj.GetPropertyValue(property.Name);

				if (value == null || valAttrib == null) continue;

				var asEnumerable = value as IEnumerable;
				if (asEnumerable != null)
				{
					List<object> items = new List<object>();
					foreach (var enumObj in asEnumerable) items.Add(enumObj);
					foreach (var enumObj in items)
					{
						result = TryValidateObjectRecursive(enumObj, results) && result;
					}
					if (items.Count < valAttrib.MinOccursOnEnumerable)
					{
						string errorMessage = valAttrib.ErrorMessage ?? "MinOccursOnEnumerable validation failed.";
						results.Add(new ValidationResult(errorMessage));
						result = false;
					}
				}
				else
				{
					result = TryValidateObjectRecursive(value, results) && result;
				}
			}

			return result;
		}
	}

	public static class ObjectExtensions
	{
		public static object GetPropertyValue(this object o, string propertyName)
		{
			object objValue = string.Empty;

			var propertyInfo = o.GetType().GetProperty(propertyName);
			if (propertyInfo != null)
			{
				objValue = propertyInfo.GetValue(o, null);
			}
			return objValue;
		}
	}
}

You need to use the ValidateObject attribute on any complex type you want validated deeply. I found out the hard way that if you try to validate all reference objects, you get nasty results on DateTime properties.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;

namespace MyValidator
{
	[AttributeUsage(AttributeTargets.Property | AttributeTargets.Field, Inherited = false, AllowMultiple = false)]
	public class ValidateObjectAttribute : Attribute
	{
		int _minOccurs = 0;
		//marker for object properties that need to be recursively validated

		public ValidateObjectAttribute() { }

		public int MinOccursOnEnumerable { get { return _minOccurs; } set { _minOccurs = value; } }

		public string ErrorMessage { get; set; }
	}
}

Of course you can take off on this or Mike’s original code and create your own validation library for a WCF service, a business logic layer, or whatever you need. Good luck and please update me on your validation adventures.

Where Agile Succeeds

In the interests of keeping the U.S. Postal Service in business, I subscribe to a number free technology periodicals. I’m too cheap to pay for anything more than MSDN Magazine. One of the free rags I like to keep in the reading room is Information Week.

A recent article entitled Where Agile Fails caught my eye. You can get a copy of it online if you want to hand over your contact info. I enjoyed the article and it makes a good general point. The author Charles Babcock draws our attention to the fact that many agile teams overlook operations and the challenges brought about by frequent releases into production.

I was only disappointed in the title. Any reader who quickly browses the article, allowing the title to influence conclusions, may walk away with the wrong idea. The title would be more informative if less typographically attractive as Where Agile Teams Fail to Involve Operations, Agile Teams Fail.

When agile development is done right, it includes as part of release and sprint planning some elements of deployment, operations, support and maintenance. Some sprints or releases planned may not go into production, so operations need not be involved too heavily, but I think they should be be apprised of the team’s plans and progress and given the opportunity to prepare in advance for the needs of the application and users.

When a release or sprint will go into production, the operations team should be represented in the planning meetings. This will give them the opportunity to learn what is expected of them and plan accordingly. It will also give them a chance to ask questions, raise concerns or even point out why they will not be able to support a go-live date for a release. And when that is the case, you want to know sooner rather than later.

Where agile succeeds is in the notion that people and interactions are more important than processes. The Manifesto does not indicate that it is referring only to developers. Clearly that is not the case. Operations people are people too. Yes, Fred, they really are.