My 2011 Software Development Reading List

A wonderful 2011 for all!  I’ve not made many New Year resolutions, but one resolution that I do want to try and keep is to work through my library of ever growing IT books that have been gathering some dust on my desk and hard-drive.  In stead of spending time reading through blogs, I want to rather concentrate on broadening/sharpening my skills through working through some of my books.  I’ve got quite a few good books where I’ve only read some part of the book and I also want to complete these as well.  So without further ado, here is my list of IT books for 2011 that I want to read:

1. [Partially Read] Release It!: Michael T. Nygard  - Excellent advise on getting and keeping your software running in production.
2. [Partially Read] Architecting Applications for the Enterprise: Dino Esposito, Andrea Saltarello – A great resource for all budding architects and a book that I want to re-read to evaluate our current architecture against some of the best practices mentioned in here.
3. [Partially Read] Continuous Delivery: Jez Humble, David Farley – Excellent advise on using build, test and deployment automation to manage your software releases.  We’ve got quite a good deployment environment at work as I’m a big fan of continuous integration, but it is always nice to read some further insights from experts to see what areas we can still improve on.
4. The Art of Application Performance Monitoring: Ian Molyneaux – Performance monitoring is high on the agenda for our Enterprise Asset Management System (EAMS).
5. Algorithms In A Nutshell: George Heineman, Gary Pollice & Stanley Selkow – Time to learn and brush up on the Graph, Search, Path finding and Network Flow algorithms.  I’ve got a sneaky suspicion that we are missing out on some opportunities to use better algorithms within our EAMS.
6. Pragmatic Thinking and Learning: Andy Hunt – Time to figure out the best way to think about solving problems.
7. Growing Software – Proven Strategies for Managing Software Engineers: Louis Testa
8. Programming Ruby 1.9 (3rd Edition): Dave Thomas – Lots of people are raving about Ruby and I want to start learning why.
9. The Ruby Programming Language: David Flanagan & Yukihiro Masumoto

Happy reading :-)

Pragma On Key – Release 3 Milestone Reached

I’m watching the final TeamCity daily build as I type this and I’m super excited about the next major milestone that we’ve accomplished for Pragma On Key, our company’s Enterprise Asset Management System (EAMS).  It’s been 30 months of hard work to re-write and extend the system using Silverlight and .NET.  We started off as early adopters with the Silverlight Beta’s and .NET 3.5 and with the latest build we are using Silverlight 3 and .NET 4.0 as the technology platform.  We evaluated Silverlight 4 but due to some unresolved memory issues in the framework we decided to stick with Silverlight 3 for now.  We hope to be upgrading to Silverlight 4 soon here after.

I’m very proud of the way the team pulled through all the difficult times to end up with what I feel is a solid platform to build on for the future.  As always there is still a lot of things we can and will improve on going forward, but I think we have come along way since the start of the project.  From adopting a new SDLC (SCRUM) to learning a complete new technology skill set (C#; Silverlight, TDD, CI etc.) and in the process close to doubling in team size. Currently we have 2 Analysts, 5 Testers, 13 Developers (me included) and a Software Development Manager.   I can honestly say that within the 2.5 years we only had one or two isolated incidents with the team dynamics.  I think this is a great testimony to the quality of the people working here at Pragma.

We are now entering the final phases of testing the new release of On Key at a big international client who is planning to go live with the system end of Feb 2011.  So after more than 20 000 CI builds, 230 tested builds, 26 820 commits and with just more than 9000 unit and integration tests it is great to finally see the new version of On Key coming of age!  We’ve got some real nifty features included in this release and I think customers will enjoy the new web based interface as well.  I hope to be blogging a bit more about this and our internal development environment in the future.  Well done to all involved!

TransactionScope Fluent Interface

During the past 2 weeks I’ve been spending a lot of time investigating the performance of Pragma On Key, our company’s Enterprise Asset Management System (EAMS).  Part of the investigation was to resolve some issues we were encountering with deadlocks occurring for some of the longer running transactions.  We are using the TransactionScope class added in .NET 2.0 to mark blocks of code in our Application Controllers as participating in a transaction.  Out of the box, the default IsolationLevel used by new a TransactionScope instance is Serializable.  This ensures the best data integrity by preventing dirty reads, nonrepeatable reads and phantom rows.  However, all of this comes at the cost of concurrency as the range lock acquired by the DBMS prevents other transactions of updating or deleting rows in the range.   So I set off to carefully consider the locking requirements for the different transactions to make sure we are acquiring the right level of locking for the different transactions.

I’ve always found the different settings of the TransactionScope class to be rather confusing.  I always need to remind myself what the different items of the IsolationLevel and TransactionScopeOption enumerations actually mean.  There is also the tricky scenario where, when passing in a new TransactionOptions instance to set a different IsolationLevel, you need to remember to set the transaction Timeout value if you are using a custom Timeout setting in your web.config file.  So I decided to create a TransactionScopeBuilder class with a fluent interface to construct new TransactionScope instances with the idea to try and hide the complexity and also provide a more intent revealing interface of what the actual TransactionScope will do.  I’ll start by showing some examples usages of the builder and then conclude with the source code for the builder itself. 

Understanding MSIL Memory Leaks

We are using a custom RuntimeDataBuilder class that dynamically constructs a .NET type in-memory using Reflection.Emit to create simple property getters/setters for the information that we receive from a generic WCF Data Service. The service exposes a "DataSet like" structure consisting out of 1..m DataTableDefinition classes each containing 1..m DataTableColumnDefinition classes. When the information is received client side, we generate the dynamic type with its property setters/getters to improve the performance and facilitate binding on our Silverlight client. All of this works fine.

I’m currently trying to figure out how the memory management for these custom dynamic types work.  I’m thinking that we might be causing a possible memory leak if we regenerate the type. Reason why I am contemplating this is that the user can change the query parameters which may then result in more/less information coming across the wire. It therefore invalidates the previous type that I created and I want to make sure that I’m are able to free up the memory used by this previous type definition as this can happen quite frequently.  From this article on MSDN we gather that if you are using Light Weight Code Generation (LCG) the code is allocated on the managed heap which will be reclaimed by the GC when there is nothing holding a reference to it. But LCG only seems to apply to dynamic methods. My concern is for the Type with all its property getters/setters that is now not required anymore. If this is allocated on the unmanaged heap our only hope for reclaiming the memory seems to be to make sure that the type is loaded into a temporary AppDomain that we can unload when it is not required anymore.  This opens up a whole new can of worms for inter AppDomain communication which I would rather avoid.

So my question is any of my readers can shed some more light on this topic.  Is my assumption correct or is there another way of reclaiming the memory?  I also posted the question to StackOverflow if you are interested in checking out the responses over there.  Thx :-)

Ultimate .NET Development Tools 2010 Edition

Here is my updated 2010 list of development tools that I prefer to use when doing .NET development.  I specifically do not include any third party control/report libraries.

Categories

• IDE = Develop/generate/refactor code within the VS IDE or separate IDE 
• SCM = Software Configuration Management (Source Control etc.)
• TDD = Test Driven Development
• DBMS = Database Management Systems
• CI = Continuous Integration
• FR = Frameworks (Persistence, AOP, Inversion of Control, Logging etc.)
• UT = Utility Tools
• CA = Code Analysis (Static + Dynamic)
• TC = Team Collaboration (Bug tracking, Project management etc.)
• MD = Modelling
• QA = Testing Tools
• DP = Deployment (Installations etc.)

 

Tools

* = free/open source

1. [IDE] Visual Studio 2010 Premium Edition
2. [IDE] ReSharper for refactoring, unit test runner and so much more
3. [IDE] CodeSmith for generating code.  Also consider T4 with Clarius’s Visual T4 Editor.  
4. [IDE]* GhostDoc for inserting xml code comments
5. [IDE] Altova Xml Suite for any xml related work.  XmlPad is the best, free alternative I know of.
6. [DBMS] SqlServer 2008 for DBMS
7. [SCM]* Subversion for source control
8. [SCM]* TortoiseSVN as windows shell extension for Subversion
9. [SCM] VisualSVN for integration of TortoiseSVN into VS.  AnkhSVN is the best, free alternative I know of.
10. [SCM]* KDiff3 for merging
11. [TDD]* NUnit as preferred xUnit testing framework
12. [TDD]* moq as mock framework.
13. [TDD] NCover for code coverage stats
14. [CI]* TeamCity as build server
15. [CI]* MSBuild Extension Pack for additional MSBuild tasks.
16. [FR]* log4net as logging framework.  Also see Log4View for an excellent UI for the log files.
17. [FR]* ANTLR and ANTLRWorks for creating custom DSL’s.
18. [FR] PostSharp as Aspect Oriented Programming framework
19. [FR]* Ninject as IoC container
20. [FR] MindScape LightSpeed as my Object-Relational-Mapper.  NHibernate is the best free alternative I’m aware of. 
21. [UT]* Reflector to drill down to the guts of any code library (also check-out the nice plug-ins)
22. [UT] Silverlight Spy to dissect any Silverlight application.
23. [UT] RegexBuddy for managing those difficult regular expressions.  Regulator is the best, free alternative I know of. 
24. [UT]* LINQPad as a easy way to query SQL databases using LINQ and as a general scratchpad application to test C#/VB.NET code snippets.
25. [UT]* Fiddler to debug all your HTTP traffic in IE.   Also see the neXpert plugin for monitoring performance problems.
26. [UT]* Firebug to assist with testing web applications running in Firefox. Also see YSlow add-on for performance testing and Web Developer add-on for additional Firefox web development tools.
27. [CA]* FxCop to enforce .NET coding guidelines
28. [CA] NDepend to get all the static code metrics I'd ever want
29. [CA] ANTS Profiler for performance and memory profiling
30. [MD] Enterprise Architect to do UML Modelling and Model Driven Design if required. Alternatively use Visio with these simple templates. 
31. [MD]* FreeMind as mind mapping tool
32. [TC]* ScrewTurn Wiki for team collaboration
33. [QA]* Eviware soapUI for functional and load testing of SOA web services
34. [QA]* Telerik WebAii Testing Framework for automated regression testing of Web 2.0 apps
35. [DP]* Windows Installer XML (WiX) for creating Windows Installers