ServiceMq Hits 10,000 Downloads

I am pleased to see this milestone of 10,000 downloads in the short history of ServiceMq and its underlying communication library ServiceWire, a faster and simpler alternative to WCF .NET to .NET RPC. And the source code for all three can be found on GitHub here.

smq10k

Over the past few weeks both libraries have been improved.

ServiceMq improvements include:

  • Options for the persistence of messages asynchronously to improve overall throughput when message traffic is high
  • ReceiveBulk and AcceptBulk methods were introduced
  • Message caching was refactored to improve performance and limit memory use in scenarios where large numbers of messages are sent and must wait for a destination to become available or received and must wait to be consumed
  • Faster asynchronous file deletion was added which eliminates the standard File.Delete’s permission demand on every message file delete
  • Asynchronous append file logging was added to improve throughput
  • The FastFile class was refactored to support IDisposable and now dedicates a single thread each to asynchronous delete, append and write operations
  • Upgraded to ServiceWire 1.6.3

ServiceWire has had two minor but important bugs fixed:

  • Code was refactored to properly dispose of resources when a connection failure occurs.
  • Previously if the host was not hosting the same assembly version of the interface being used, the connection would hang. This scenario now properly throws an identifiable exception on the client and disposes of the underlying socket or named pipe stream.

Real World Use

In the last month or so, I have had the opportunity to use both of these libraries extensively at work. All of the recent improvements are a direct or indirect result of that real world use. Without disclosing work related details, I believe it is safe to say that these libraries are moving hundreds of messages per second and in some cases 30GB of data between two machines in around three minutes across perhaps 300 RPC method invocations. Some careful usage has been required given our particular use cases in order to reduce connection contention from many thousands of message writer threads across a pool of servers all talking to a single target server. I’ve no doubt that a little fine tuning on the usage side may be required, but overall I’m very happy with the results.

I hope you enjoy these libraries and please contact me if you find any problems with them or need additional functionality. Better yet, jump onto GitHub and submit a pull request of your own. I am happy to evaluate and accept well thought out requests that are in line with my vision for keeping these libraries lightweight and easy to use.

One other note

I recently published ServiceMock, a tiny experimental mocking library that has surprisingly been downloaded over 500 times. If you’re one of the crazy ones, I’d love to hear from you and what you think of it.

DomainAspects 2.0: Aspect Oriented Library with Generic WCF Host

Would you like to just focus on your business domain and host it anywhere easily rather than fussing with infrastructure? With DomainAspects you just write the interface in a project called [MyDomain].Common and the implementation is a project called [MyDomain]. Now add a brief and easy configuration string to both client and service host and xcopy deploy. Restart your hosting service if you are using the domain remotely.

With DomainAspects, all access to the implementation is now controlled as you have specified through easy attributes on your implementation without ever writing specific code to validate and authorize. Each and every call is logged without adding any logging code. Each and every exception is logged without ever adding a try/catch block in your code. And exceptions are captured, logged, and thrown up the chain to the client.

Introducing the 2.0 version of DomainAspects. This new version contains some major changes, the chief among them is the introduction of a WCF host container that allows you to access your aspect oriented domain layer remotely. More on the changes and features later, but first, let’s review what has come before and contributed to this new release:

And some earlier posts that have formed the creation of the WCF host in this new version:

DomainAspects 2.0 Introduced Today

Over the last couple of months, I’ve spend considerable time on the weekends to complete the idea of merging my previous work on a generic WCF host into my DomainAspects project. Since I had not touched either of these projects for some time, it was fun and interesting to look at them from a new perspective. There are many changes for the better here. Please download it and give it a try. I am eager to hear your feedback.

New in DomainAspects 2.0

Today, I’m releasing DomainAspects 2.0 which supports the following new features:

  • A WCF host service for accessing domain services running in another process or server with the following features/benefits:
    • Fast and encrypted net.tcp binding with Windows permissions.
    • Simplified custom configuration section that is the same on client and server.
    • Custom instance provider uses the DomainAspects service factory and Windsor for instance creation and release.
    • Ability to specify service instance lifestyle closely matching Windsor (PerThread, Pooled, Singelton, SingleCall).
    • Service host that will run in console mode for easy debugging.
    • Flip a config switch and install the service host as a windows service. No recompile required.
    • RemoteClient implements IDisposable correctly, dealing with Abort vs Close for you.
    • Host can be configured to use a fake PrincipalProvider for easily testing a variety of user roles and combinations.
    • Reliance on DomainAspects for authorization rather than configured local Windows groups.
  • A light remote client assembly allows access DomainAspects with minimal deployment requirements.
  • Local client and remote client both return an instance of the service interface, allowing a domain to be moved to another process or remote server with minimal changes to client code.
  • Upgrade to the current stable release Castle Windsor 3.0 container libraries.
  • Improved domain service factory allows greater instance creation control.
  • Additional tests, including multi-threaded remote tests focusing on instance creation and pooling.
  • More intuitive, self-describing classes to make understanding the code easier.

Using DomainAspects 2.0

Using the DomainAspects solution is easy. You must have Visual Studio 2010 Pro or higher. If you do not, you may spend considerable time trying to get it running. I would not recommend it.

Download the code and make sure you right click the zip file and go to properties and unlock the zip file before you extract it.

To run remote tests, be sure to start the DomainAspects.Host without debugging first. I recommend running the MultiRemoteTests class using a multiple project startup with the host running first and the DomainAspects.TestConsole running the test rather than the automated tests runner since I have not found a way to get those tests to run properly within the test harness.

Now run the tests. Watch the host console as the MultiRemoteTests run. Note the instance creation on second and third run take zero time because these instances have been pooled.

When you are ready to build your own domain service, just follow the examples in the TestDomain and TestDomain.Common projects. Configure the host and the client as shown in the app.config files in the included projects appropriately to match your new domain and domain.Common assemblies.

If you are going remote, build and deploy the DomainAspects.Host. Make sure you switch the “consoleMode” attribute in the config on the server side to “Off” so that it will install and run as a service.

Download DomainAspects20.zip (2.10 mb)

Custom Recursive Model Validation in .NET Using Data Annotations

Need to write your own model validator outside the scope of an application framework such as ASP.NET MVC? A short while ago, I needed to do just that. I was writing a WCF service with a relatively complex data model which required a much greater level of validation than the DataMember attribute’s IsRequired property could provide.

Here’s the solution I found. But first a bit of background.

Validation in ASP.NET MVC 
I’ve been using Data Annotations attributes for view model validation for years in the context of ASP.NET MVC for model validations, client and server side. I always took the server side validation for granted and looked at the client side with greater interest. When getting started with ASP.NET MVC, I used Steve Sanderson’s xVal library. Then I switched to ASP.NET MVC 3’s client side validation.

For client side validation, I’m really starting to like jQuery’s unobtrusive validation and the ASP.NET MVC’s HtmlHelper class and its GetUnobtrusiveValidationAttributes method. And for the server side, view model binding and the ModelState.IsValid property works fine.

Validation Without an App Framework 
But this post is not about client side validation or server side validation in ASP.NET MVC. I had to do the validation against the model in the WCF service which, as far as I know, does not have a nice model validation facility other than its own data model serialization which just throws a nasty exception should the data being passed over the wire not provide required data.

So why not use the same approach used by our friends in the ASP.NET MVC world. A little Google investigation turned up something that looked like exactly what I wanted. I found an answer posted to StackOverflow by Mike Reust which included a link to his DataAnnotationsValidatorRecursive library. (Gotta love StackOverflow.)

After some experimentation, I had made some tweaks to Mike’s code and came up with something that worked the way I wanted it to. Here’s a partial example of a data model and how the ValidateObject with it’s optional MinOccursOnEnumerable can be used to require one or more items to be included in a required complex enumerable type that will get recursively.

[DataMember(IsRequired = true), 
	Required(ErrorMessage = "Employer cannot be null."), 
	ValidateObject]
public Employer Employer { get; set; }


[DataMember(IsRequired = true), 
	Required(ErrorMessage = "Children cannot be null."), 
	ValidateObject(MinOccursOnEnumerable = 1)]
public Child[] Children { get; set; }

Using the recursive validation library is easy. Here’s an example:

//perform validation using DataAnnotations for custom validation messages
List<ValidationResult> results = new List<ValidationResult>();
bool isValid = DataAnnotationsValidator.TryValidateObjectRecursive<MyModelData>(model, results);

//now examine isValid and the results

And here’s the code for the validation extensions and ValidateObject attribute.

using System.Collections;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System;

namespace MyValidator
{
	public static class DataAnnotationsValidator
	{
		public static bool TryValidateObject(object obj, ICollection<ValidationResult> results)
		{
			return Validator.TryValidateObject(obj, new ValidationContext(obj, null, null), results, true);
		}

		public static bool TryValidateObjectRecursive<T>(T obj, List<ValidationResult> results)
		{
			bool result = TryValidateObject(obj, results);

			var properties = obj.GetType().GetProperties().Where(prop => Attribute.IsDefined(prop, typeof(ValidateObjectAttribute)));

			foreach (var property in properties)
			{
				var valAttrib = property.GetCustomAttributes(typeof(ValidateObjectAttribute), true).FirstOrDefault() as ValidateObjectAttribute;
				var value = obj.GetPropertyValue(property.Name);

				if (value == null || valAttrib == null) continue;

				var asEnumerable = value as IEnumerable;
				if (asEnumerable != null)
				{
					List<object> items = new List<object>();
					foreach (var enumObj in asEnumerable) items.Add(enumObj);
					foreach (var enumObj in items)
					{
						result = TryValidateObjectRecursive(enumObj, results) && result;
					}
					if (items.Count < valAttrib.MinOccursOnEnumerable)
					{
						string errorMessage = valAttrib.ErrorMessage ?? "MinOccursOnEnumerable validation failed.";
						results.Add(new ValidationResult(errorMessage));
						result = false;
					}
				}
				else
				{
					result = TryValidateObjectRecursive(value, results) && result;
				}
			}

			return result;
		}
	}

	public static class ObjectExtensions
	{
		public static object GetPropertyValue(this object o, string propertyName)
		{
			object objValue = string.Empty;

			var propertyInfo = o.GetType().GetProperty(propertyName);
			if (propertyInfo != null)
			{
				objValue = propertyInfo.GetValue(o, null);
			}
			return objValue;
		}
	}
}

You need to use the ValidateObject attribute on any complex type you want validated deeply. I found out the hard way that if you try to validate all reference objects, you get nasty results on DateTime properties.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;

namespace MyValidator
{
	[AttributeUsage(AttributeTargets.Property | AttributeTargets.Field, Inherited = false, AllowMultiple = false)]
	public class ValidateObjectAttribute : Attribute
	{
		int _minOccurs = 0;
		//marker for object properties that need to be recursively validated

		public ValidateObjectAttribute() { }

		public int MinOccursOnEnumerable { get { return _minOccurs; } set { _minOccurs = value; } }

		public string ErrorMessage { get; set; }
	}
}

Of course you can take off on this or Mike’s original code and create your own validation library for a WCF service, a business logic layer, or whatever you need. Good luck and please update me on your validation adventures.

WCF Plugin Architecture for .NET Windows Service with Easy Debug Console Mode

I've been wanting a simplified WCF plugin architecture for .NET Windows service with easy debug console mode to make my life easier. Building services with Windows Communication Foundation (WCF) is simple and hard. The service and data contract code is simple, like this:

namespace Test2Common	
{
  [ServiceContract]
  public interface IMyOtherService
  {
    [OperationContract, FaultContract(typeof(WcfServiceFault))]
    string GetName(string seed);

    [OperationContract, FaultContract(typeof(WcfServiceFault))]
    Person GetPerson(int id);
  }

  [DataContract]
  public class Person
  {
    [DataMember]
    public string Name { get; set; }

    [DataMember]
    public string Title { get; set; }
  }
}

But that is where simple ends. The hosting and client proxy code and the configuration XML (or code) is not simple. Even when I use tools to generate these artifacts, I find myself having to tweak and fix and delve deeper than time permits. And since I prefer simple, I decided to create a reusable framework for hosting my limited scope services. Here's what I wanted (download PluggableWcfServiceHost.zip (38.05 KB)):

  • Fast net.tcp binding
  • Windows credentials and authentication
  • Encrypted and signed transport (no packet sniffing allowed)
  • Simplified configuration (hide everything I don't want to see)
  • Windows Service host that behaves like a Console app when I'm debugging
  • Dynamic loading of the service (no changes to the host code to add a new service)
  • Generic client so I don't have to write or generate proxy code
  • Client that is truly IDisposable (hide Abort vs Close for me)
  • Long timeout in DEBUG mode so I can really take my time while debugging
  • Inclusion of exception details in DEBUG mode only
  • Base service class with a simple Authorize method to support multiple Windows groups
  • Support for multiple Windows group authorization
  • Identical configuration for server and client
  • Cached resolution of service plugin service and contract types
  • Minimal number of assemblies (projects) in the solution
  • Keep the implementation of the service hidden from the client
  • (Probably some I've forgotten to mention)

Here's the simplified config with two services configured:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
    <section name="wcfServices" 
          type="WcfServiceCommon.WcfServiceConfigurationSection, WcfServiceCommon" />
  </configSections>
  <appSettings/>
  <connectionStrings/>
  <wcfServices consoleMode="On">
    <services>
      <add key="test1"
          serviceAddressPort="localhost:2981"
          endpointName="Test1EndPoint"
          authorizedGroups="WcfServiceClients,someOtherGoup"
          hostType="Test1Service.ThatOneService, Test1Service"
          contractType="Test1Common.IThatOneService, Test1Common" />
      <add key="test2"
          serviceAddressPort="localhost:2981"
          endpointName="Test2EndPoint"
          authorizedGroups="WcfServiceClients,someOtherGoup"
          hostType="Test2Service.MyOtherService, Test2Service"
          contractType="Test2Common.IMyOtherService, Test2Common" />
    </services>
  </wcfServices>
</configuration>

And here's the implementation of the service (It really is simple):

namespace Test2Service
{
  public class MyOtherService : WcfServiceBase, IMyOtherService
  {
    public string GetName(string seed)
    {
      base.Authorize();
      return "This is my name: " + seed.ToUpper();
    }

    public Person GetPerson(int id)
    {
      base.Authorize();
      return new Person { Name = "Donald Trumpet", Title = "King of the Hill" };
    }
  }
}

Now for the really fun part. The client. Hey, where's the proxy? Hidden away just like it ought to be.

namespace WcfSvcTest
{
  class Program
  {
    static void Main(string[] args)
    {
      using (var client1 = WcfServiceClient<IThatOneService>.Create("test1"))
      {
        Console.WriteLine(client1.Instance.GetName("seed"));
        var add = client1.Instance.GetAddress(8);
        Console.WriteLine("{0}", add.City);
      }

      using (var client2 = WcfServiceClient<IMyOtherService>.Create("test2"))
      {
        try
        {
          Console.WriteLine(client2.Instance.GetName("newseed"));
          var per = client2.Instance.GetPerson(7);
          Console.WriteLine("{0}, {1}", per.Name, per.Title);
        }
        catch (FaultException<WcfServiceFault> fault)
        {
          Console.WriteLine(fault.ToString());
        }
        catch (Exception e) //handles exceptions not in wcf communication
        {
          Console.WriteLine(e.ToString());
        }
      }

      Console.ReadLine();
    }
  }
}

To save space, I only wrapped the use of the client in try catch on the second test service.

Note: You have to remember that the WcfServiceHost requires the "common" and the "service" assemblies of your dynamically loaded services in it's bin folder. The client (see WcfSvcTest project in the solution) will also need a copy of the "common" assemblies in it's bin folder. You'll find I'm doing that for the test using post-build commands (copy $(TargetPath) $(SolutionDir)WcfServiceHost\bin\debug\). And of course, both need to have identical config sections as shown in the code.

I hope you find this WCF plugin architecture and the accompanying code useful. I will certainly be using it. If you do use it, please let me know how it goes.

Download PluggableWcfServiceHost.zip (38.05 KB)