n.code

SQL Server Error: OLE DB error trace [OLE/DB Provider 'SQLOLEDB' IUnknown::QueryInterface returned 0x80004005: The provider did not give any information about the error.]

n.code — Mon, 20 Aug 2007 09:15:00 GMT

SUMMARY You receive the following error trying to execute a query against a linked server: Server: Msg 7399, Level 16, State 1, Line [Line Number] OLE DB provider 'SQLOLEDB' reported an error. The provider did not give any information about the error. OLE DB error trace [OLE/DB Provider 'SQLOLEDB' IUnknown::QueryInterface returned 0x80004005: The provider did not give any information about the error.] Solution Turn off the SQLOLEDB provider options LevelZeroOnly IndexAsAccessPath...(read more)

Random Ramblings Over Ruby On Microsoft .NET

n.code — Fri, 03 Aug 2007 12:57:00 GMT

Martin Fowler argues that IronRuby gives Microsoft the opportunity to collaborate with, and extend its .NET platform to a larger AlphaGeeks and open source developer base. According to Martin the Ruby community is very eager to develop on the Microsoft platform.

One thing that caught my attention was his argument that there is a concern that AlphaGeeks are moving away from the Microsoft platform. His contacts at Redmond told him that they were seeing a departure of technical leaders away from the Windows platform. Yes a lot of developers don’t like Microsoft’s approach to software and development (I’m a big critic myself), and I’m sure those will leave the MS camp when they feel it’s necessary. But a lot of AlphaGeeks started using C#, and got a new eagerness to use it since its inception.

From my experience Microsoft was bleeding leading edge developers to Java during the days when they were still trying to figure out what to make of it (who remembers the J++ dispute), and how it will put a new spin on the software development world. But since the advent of C# and .NET, they’ve pretty much delivered the goods.

Herewith my random ramblings that I sent to Martin:

1. Martin.

”Another reason is people. Microsoft doesn't like to acknowledge this in public, but there is a real concern that AlphaGeeks are moving away from the Microsoft platform. There's a growing sense that Microsoft's vision is armies of Morts in command-and-control organizations. There often seems to be outright discouragement of tools to enable talented enterprise developers, or of agile development processes.

A few years ago my (limited) contacts in Redmond told me that they were seeing a real drift of technical leaders away from the Windows platform. More recently these signs seem to be increasing. Reading the 'softie part of my blogroll I got a sense of real disillusionment amongst people who have been long-time Microsoft supporters. Agile-oriented developers have been frustrated with the direction of Microsoft tools. Microsoft conferences barely mentioning agile processes, leaning much more to waterfall approaches. The tools, with their rigid role-separations, actively discourage the blurry boundaries that agilists prefer."

N.Code.

Ever considered that this might exactly be the reason that organizations are increasingly adopting their use of .NET? Think about it. There are a lot more average developers (possibly myself) available. This rigid team and project structure suits companies, and average developers. Managers know what they can expect of their teams, and the "insert role X here" knows exactly what is expected of him. Each one almost like a small gear in the great development machine.

I mean a manager doesn't want to know that he is dependent on a talented agilest. The ability to just plug in a new employee, once the previous one's lifetime expires, exceeds the desire to apply multi-talented ones. Developers are approached as commodities.

Hiring talented developers also costs more. This is probably one of the biggest complaints I hear about non-Microsoft platforms, that developers for them are too difficult to find, and costly to hire, compared to the masses of Microsoft developers.

Of course this is more from the angle of IT departments in non-IT organizations, like Banks, and the government.

2. Martin.

"Indeed it's already past next year for Microsoft. We've seen a noticeable drop-off in interest from our clients for Microsoft projects, particularly in the US. In Australia, .NET hasn't got any foothold at all amongst our clients. I'm not sure what to make of this data. We aren't so big to be a statistically valid sample on our own. But it's a useful data point nonetheless particularly since we like to think our clients are the "alpha IT shops".

N.Code.

Intervate mainly does work for non-IT organizations, and I do not see any slowdown in .NET adoption. Obviously my perception is skewed, since Intervate is a Microsoft partner. But still, we have done work in almost every big South African organization, and I have yet to see an anti-Microsoft one. Even the South African government touting its OSS strategy, is currently one of the biggest Microsoft shops around, and many new projects are done in .NET.

3. Martin.

"This is important because, as Tim O'Reilly believes, the alpha geeks point to what everyone else will be doing in a few years time."

N.Code.

Examining how most events in history ended, it was always the majority/masses that had the final say. Yes the royals always ruled for a while, but eventually the revolution brought them to their knees. The Microsoft platform has always been the home of the majority of people (developers?).

4. Martin.

"So what I see for Ruby and Microsoft is an opportunity."

N.Code.

There is definitely the opportunity for Microsoft here to win the hearts and minds of the OSS and alpha geeks to a certain extent. But there's an additional reason for Ruby, and that is to gain a bigger share of Microsoft based development, and gain the recognition of Microsoft shops. This will give Ruby a Microsoft facing side, that does not seem as threatening as its current only-for-OSS image. It will allow Ruby to take part in the growth Microsoft based solutions.

Common .NET Memory Issues

n.code — Mon, 25 Jun 2007 15:29:00 GMT

Charl Botha, senior developer at Intervate, and I had some discussions around .NET memory issues, and how to measure the size of a CLR object in memory.

Here's what Charl had to say:

"I have yet to discover how one can retrieve the size of an object in pure CLR, the method that I used was to take an object created in CLR, get the GCHandle, then get the size of the object in unmanaged memory. This I don't think would help you.

There is software available that you can use, free, that can measure memory usage.

They are dead slow since they are virtual machines, like Java, but are able to take the measurements for you.

The CLR profiler I found to be the best product to help me find out which objects are not memory happy.

Here's a list of things that are not very good to memory and will abuse you system.

Note, this infromation was composed nearly two years ago based on the 1.1 .Net framework, so I'm not 100% sure that they still affect the system as badly as they did last time.

1. Delegates, the typical += assigning to events, a lot of coders abuse this and every time a new object is created they create a new delegate and bind it to the object, bad.

There should only be one delegate created when the assembly is loaded and reused, the -= must then be used to unbind the object when the destructor is called or disposed. Very, very important.

2. Secondly, fonts, brushes, and graphics in general. Each graphically driven object needs a handle, and handles are created when the object is created. But when the GC collects the object, it does not dispose of the handle, bad.

Once again, use the object that are already created by the application and not manually by your code.

3. Streams, any stream reader or writer must be closed and disposed, if a coder does not do this, the handle to the memory is kept open until the process is killed. Always, close stream, so search through your code and see where you open streams, from files, to XML, to serialization.

4. Threads, every time a thread is created in the virtual machine, certain memory is allocated to control this thread, in the real world, i.e. OS systems 5+ allocate 1mb of memory to manage a thread, create a lot of threads, use a lot of memory, whenever possible use thread managers and processing queues to run transactions rather than just creating a thread for every transaction.

5. Lastly, use the GC.WaitForPendingFinalizers after transactions, this will help with thread memory allocation not just application memory, remember, threads drive process, processes, drive jobs."

Hope this discussion helps others, as it helped me with some commonly encountered .NET memory usage and development issues.

Bulk Update and Insert of Object State Stored by SQL Server 2000, Using .NET (C#)

n.code — Mon, 14 May 2007 17:19:00 GMT

This is a how to/quick start for doing bulk inserts or updates of data on SQL Server 2000 and .NET.

1. Create a simple class:

    using System;
    ...
    ...

    [Serializable]
    public class Entity
    {
        private int _id;
        private string _name;
        private string _description;

        public int ID
        {
            get
            {
                return _id;
            }
        }

        public string Name
        {
            get
            {
                return _name;
            }
            set
            {
                _name = value;
            }
        }

        public string Description
        {
            get
            {
                return _description;
            }
            set
            {
                _description = value;
            }
        }
    }
    ...
    ...

2. Create a class to manage a set of Entities:

    ...
    ...

    using System;
using System.Data;
    using System.Collections;
    using System.Data.SqlClient;
    using Microsoft.Practices.EnterpriseLibrary.Data;
    using Microsoft.Practices.EnterpriseLibrary.Data.Sql;

    public class Entities: IDisposable
    {
        private ArrayList _entityList = new ArrayList();

        public Entity this[int idx]
        {
            get
            {
                return _entityList[idx] as Entity;
            }
        }


        public void Add( Entity entity )
        {
            lock ( _entityList.SyncRoot )
            {
                _entityList.Add( entity );
            }
        }


        public void Dispose()
        {
            _store();
        }

        private void _store()
        {
            ArrayList dirtyEntityList = new ArrayList();

            lock ( _entityList.SyncRoot )
            {
                for ( int iEnt = 0; iEnt < _entityList.Count;   iEnt++ )
                {
                    Entity entity = _entityList[iEnt] as Entity;

                    if ( entity.IsDirty )
                    {
                        dirtyEntityList.Add( entity );
                    }
                }
            }

            // Copy ArrayList to Entity array
            Entity[] entities = dirtyEntityList.ToArray( typeof( Entity ) ) as Entity[];

            // Serialize array
            string xml = Serialization.XmlSerialize( entities );

            // Create SQL command
            Database db = DatabaseFactory.CreateDatabase();
            DBCommandWrapper command = db.GetStoredProcCommandWrapper( "Entity_Set" );
            command.AddInParameter( "@entities", DbType.String, xml );

            // Send dirty Entities to SQL Server
            db.ExecuteNonQuery( command );
        }
       }

    ...
    ...

3. Create a method to serialize objects:

    using System;
    using System.IO;
    using System.Text;
    using System.Xml.Serialization;

    ...
    ...

    /// <summary>
    /// Gets the XML of the passed object.
    /// </summary>
    public static string XmlSerialize( object obj )
    {
        // Validate argument 'obj'
        if ( obj == null )
            throw new ArgumentNullException( "obj", "Argument 'obj' is null" );

               XmlSerializer ser = new XmlSerializer( obj.GetType() );

        MemoryStream ms = new MemoryStream();
        UTF8Encoding utf8 = new UTF8Encoding(false,false);
        StreamWriter sw = null;
        string xml = null;

        using ( ms )
        {
            using ( sw )
            {
                sw = new StreamWriter( ms, utf8 );
                ser.Serialize( sw, obj );
                xml = utf8.GetString( ms.GetBuffer() );
            }
        }

        return xml;
    }

    ...
    ...

4. Create a stored procedure that will de-serialize the object’s XML into a table, from which new and current records are inserted or updated:

   CREATE PROCEDURE dbo.Entity_Set
   ( @entities   VARCHAR( 5000 ) )
   AS

   SET NOCOUNT ON

   DECLARE @xml INT
         EXEC SP_XML_PREPAREDOCUMENT @xml OUTPUT,

   @entities

   -- Insert new entities
    INSERT INTO    Entity
                            ( Name
                            , Description )
    SELECT            XMLEntity.Name,
                          XMLEntity.Description
    FROM            OPENXML ( @xml, ‘//Entity’, 2 )
    WITH               ( ID   INT
                           , Name   VARCHAR( 50 )
                           , Description VARCHAR( 100 ) )
                           XMLEntity
     WHERE          XMLEntity.ID = 0

   -- Update current entitites
   UPDATE           Entity
    SET                Name = XMLEntity.Name,
                          Description = XMLEntity.Description
     FROM             OPENXML ( @xml, ‘//Entity’, 2 )
    WITH               ( ID       INT
                            , Name       VARCHAR( 50 )
                          , Description    VARCHAR( 100 ) )
                            XMLEntity
   WHERE           XMLEntity.ID = Entity.ID

   EXEC SP_XML_REMOVEDOCUMENT @xml

   GO

The OpenXml method’s last parameter indicates that this is an element centric XML table. .NET by default serializes objects with an element based schema. Use 0 or 1 for attribute centric XML documents.

5. Now create a NUnit test class, to demonstrate the new bulk insert and update of data:

    /// <summary>
    /// Summary description for EntityTests.
    /// </summary>
    [TestFixture]
    public class EntityTests
    {
        private static Random _random = new Random();

        [Test]
        public void Entities_Update_Test()
        {
            using ( Entities ents = new Entities() )
            {
                ents.Add( _generate() );
                ents.Add( _generate() );
                ents.Add( _generate() );
                ents.Add( _generate() );
            }
        }

        private static Entity _generate()
        {
            Entity ent = new Entity();
            ent.Name = _random.NextDouble().ToString();
            ent.Description = _random.NextDouble().ToString();
            return ent;
        }
    }

Run Entities_Update_Test() in NUnit, and see how SQL Server bulk updates the Entity table in one go!

.NET Memory Leak: XmlSerializing your way to a Memory Leak

n.code — Thu, 19 Apr 2007 08:45:00 GMT

Tess Ferrandez talks about a .NET memory leak, caused by using the default constructors other than, XmlSerializer(type) and XmlSerializer(type, defaultNameSpace), when XML serializing objects. In a nut shell, dynamic code generation, which is used in XML Serialization, Regular Expressions, and XSLT transformations, can lead to memory leaks.

Setting up MySQL for Python (MySQLdb) on Mac OS X

n.code — Wed, 31 Jan 2007 12:55:00 GMT

MySQL is an excellent, very popular and open source database management system. A binary installation is available for Mac OS, and the process of installing MySQL on Mac OS is also very well documented. So just head over to MySQL.com, where you will find the binary installation, together with extensive installation, and usage documentation.

To connect to MySQL from Python use MySQL for Python (MySQLdb). Get MySQLdb from http://sourceforge.net/projects/mysql-python/.

Download MySQL from mysql.com and install the DBMS as instructed in the documentation.
Make sure “Library/Python/2.3” does not have:

Directory “MySQL”

File “_mysql.pyd”

File “_mysql_exceptions.py”

File “_mysql_exceptions.pyc”

Either uninstall MySQL for Python if a setup program was used to install it, otherwise manually delete them.

Download and unpack MySQL for Python. At the time of writing this is MySQL-python-1.2.1_p2.tar.gz .
Open Terminal and change to the directory where MySQLdb was unpacked to.
Get rid of any previous builds that might interfere, by deleting the “build” directory if it exists.
Edit the setup.py file, and change:

    return popen(“mysql_config --%s” % what)
                to
    return popen(“/usr/local/mysql/bin/mysql_config --%s” % what)
Cleanup any previous install attempts:

python setup.py clean
Build MySQLdb:

python setup.py build

Viola! MySQL for Python is all setup, and ready for your Python-MySQL data access code. Enjoy!

.NET How To Create and Use Custom Name-Value Config Sections

n.code — Fri, 19 Jan 2007 17:05:00 GMT

Making over-use of the .NET's AppSettings quickly results in a messy configuration file. Especially when several assemblies' configurations are combined, it's quite painful trying to figure out which settings are for which assemblies. Some of the assemblies might even use the same setting keys, causing a real configuration nightmare. Therefore to keep things neat and tidy, without a lot of extra effort, create your own name-value configuration groups and sections, using the sectionGroup and NameValueSectionHandler.

Declare a new sectionGroup:
<configSections>
    <sectionGroup name="shinobido.util">
       ...
    </sectionGroup>
</configSections>
Add a section to the sectionGroup:
<configSections>
    <sectionGroup name="shinobido.util">
       <section name="shinobido.util.auditLogging" type="System.Configuration.NameValueSectionHanlder" />
    </sectionGroup>
</configSections>
Add the new configuration group and section using the named elements. Inside configuration section "add" elements, with key-value attributes store settings, just like AppSettings.
<shinobido.util>
    <shinobido.util.auditLogging>
       <add key="applicationName" value="BlackSun" />
       <add key="cached" value="true" />
    </shinobido.util.auditLogging>
</shinobido.util>
Retrieve the settings using ConfigurationSettings.GetConfig(...), and cast the return value to a NameValueCollection.

using System.Configuration;
using System.Collections.Specialized;

public class Application
{
    private static string _name;
    ...

    public static Name
    {
       get
       {
              if ( _name == null )
                {
                    NameValueCollection settings = ConfigurationSettings.GetConfig( "shinobido.util/shinobido.util.auditLogging" )
                                                                    as NameValueCollection;
                    _name = settings["applicationName"];

                    if ( _name.Length <= 0 ) throw new ApplicationException( "Application name is not configured." );
                }

                return _name;
       }
    }
}

That's it. Your own custom config-section in 60 seconds.