I'm busy setting up a new BizTalk environment for DR purposes and decided it is time to see how far I can automate the process. I discovered someone (Romiko) already created a utility that helps in this process (Automating Hosts, Host Instances and Adapter Handlers configuration in BizTalk 2006) but it needed it to be a bit more flexible and configurable. Thus I made some changes to the code - adding a few bits, editing some others.
The tool uses WMI so it is not BizTalk version specific. I Tested on BizTalk 2009 and ran it on BizTalk 2006 (R1) without problems.
Basically what this tool does is set up Hosts, Host Instances and Adapter handlers using a simple XML file you can edit separately. The structure of the xml file is pretty straight forward. e.g.
<?xml version="1.0" encoding="utf-8"?>
<BtsAdminConfiguration>
<Hosts>
<Host hostname="FileReceive" ntgroupname="BizTalk Application Users" isdefault="false" hosttracking="false" authtrusted="false" hosttype="1" ishost32bitonly="False"/>
<Host hostname="FileSend" ntgroupname="BizTalk Application Users" isdefault="false" hosttracking="false" authtrusted="false" hosttype="1" ishost32bitonly="False"/>
</Hosts>
<HostInstances>
<HostInstance servername="." hostname="FileReceive" username="" password="" startinstance="true"/>
<HostInstance servername="." hostname="FileSend" username="" password="" startinstance="true"/>
</HostInstances>
<Adapters>
<Adapter name="FILE" type="FILE" comment="FILE adapter">
<ReceiveHandler hostname="FileReceive"/>
<SendHandler hostname="FileSend"/>
</Adapter>
</Adapters>
</BtsAdminConfiguration>
Changes I made
1. I added support for 64-bit Hosts which weren't in the original.
2. made it possible to choose if you want to remove previous configured bits.
3.
I changed it so you don't have to go and save passwords in the XML file
(possible security risk) and pass this password as a parameter at
run-time - to be used globally. The same was done for the username.
I did notice that it seems the adapter handlers do not get properly removed if you choose to do that but since my focus is on creating and setting these up I'm ignoring the problem.
Perhaps this tool wil help other BizTalk administrators/developers as well.
One of those nasties when dealing with BizTalk BAM is setting up the BAM portal. When you run on a 64-bit machine there are all kind of 'fun' things to remember to do (above all the other 'fun' things to set it up...). For some obscure reason BAM portal must run in 32-bit and this is not the case if you are running on a 64-bit platform. Hence you must enable 32-bit mode for IIS. To do so run the following command:
cscript.exe c:\Inetpub\AdminScripts\adsutil.vbs set
W3SVC/AppPools/Enable32BitAppOnWin64 1
I really hope the BizTalk team at MS will one day fix all this crap (like the ExplorerOM that also only supports 32-bit).
Never a dull day in BizTalk land...
Yesterday I installed the developer edition of the new BizTalk server 2009 on my work machine. Except for the one dependency on ADO.Net (ASADOMD10) everything for the install/upgrade went fine. It seems not too much have change as far as functionality goes. Remember I'm doing mostly administering and support on the system so development changes are not my focus. Even in the development area it seems most things have stayed the same (general stuff like creating schemas, maps, orchestrations etc.). At least the Admin console loads 'fantastically' faster - being async now I guess.
But I do have one big thing I'm unhappy with! They removed the damn refresh button from the Admin console's toolbar! I mean, why??? Yes, this seems like a stupid thing to complain about but makes updating the details in the tool hard at times if you have to click on multiple places plus right-click and then select "Refresh' from a context menu. F5 does not always work. If the focus happens to be on the 'Group overview page' instead of the tree view node to the left then F5 does nothing sometimes despite the message on this page that states "Refresh this page to update result (press F5). You have to click on the 'Group Hub' tab for F5 to work.
At least now we use VS2008 and SQL2008 and can dump the older versions (VS2005 and SQL2005). Having to run both versions of VS/SQL always dragged the machine down consuming double the resources. I've been using VS2008 and SQL2008 for quite some time and tried getting away from the previous versions whenever I could. Also, now we have Unit testing for BizTalk projects (not enabled by default) - I have not tried this out yet.
If you haven't noticed it yet HAT is gone - not that some people would notice as they never used it. BAM is.. well still a pain in the behind to set up initially - especially the Excel Add-in that does not install properly if you do an upgrade on an existing 2006R2 install. Then there is the ExplorerOM issue... still only 32bit support!
All in all BizTalk 2009 isn't too bad except that it is not much better than BizTalk 2006R2 ;) The only real improvements are the VS2008/SQL2008 bits but otherwise there is not too much that makes it better than R2 - mind you, I haven't looked at the unit testing stuff or TFS stuff yet - but that is only really helpful for developers and not support people.
I had the need to see what tracking options were enabled in multiple BizTalk environments (and even different versions). You could go into the BizTalk admin tool and select 'All Artifacts' and the select multiple objects but that does not tell you the details of different options were selected for different objects.
Initially I was looking at plain WMI or direct database queries but then settled using the Explorer OM API. The only real big pain of ExplorerOM is that it only supports 32bit environments and we happen to have more than just a few 64bit machines for BizTalk. Luckily Visual Studio supports the Platform Target option so you can select to only support 32bit. It is a real shame Microsoft haven't updated BizTalk's ExplorerOM to support 64bit as well. I think they deserve some flack over that... ;)
So basically I created a simple C# winform app that takes the BizTalk management database server and database name and then creates a single view 'report' that you can save if you want to. The application itself is actually very simple but it can of course be extended to do alot of other things as well.
I sometimes get the feeling that there are some wonderful bits in the BizTalk 'world' but somehow they were never quite finished or polished off. The ExplorerOM API provides some wonderful functionality but then (for example) they neglected to update it to support 64bit. Then the fact that dispite the fact that BizTalk 2006/R2 and 2009 are so much alike you cannot administer older versions from the newer APi - something that the SQL server guys have managed to do for ages.
Anyway my next idea is to create something with which I can configure tracking from batch files - but on steroids using the ExplorerOM. Sometimes you need to enable or disable tracking groups of objects.
I'll attach the source for the viewer app. Comments are welcome.
This is just a mind tester - nothing serious.
In the following number sequence, what number is 'missing'
2, 6, 15, 77, 143, 221
Try this without cheating (aka Googling)
No seriously, I've created something that suck up stuff (messages in BizTalk) and throw them away! Some people might say but BizTalk does that out of the box!... but that is not true (most of the time) - especially if you're a beginner with the product it might seem that way...
But on a serious note, I had the need to suppress some 'unwelcome' or unneeded messages since some systems were sending messages up to our HO that should not be created - yes, bad design etc. but we're stuck with it now and fixing it is not easy as there are hundreds of POS (point of sales) that are located all over the country and cannot be updated quickly due to all the red tape... lets no go into that further.
My solution is to simply create a send pipeline component that deletes the entire message stream - actually everything about the message. Normally you would do some manipulation inside the pipeline component like reformatting the message stream or do other stuff like database look ups to supplement something. This pipeline component is then simply placed inside a send pipeline that can then be used in a send port that subscribes to the required messages. In our case it must filter out only some messages as all messages arrives through the same receive port.
Perhaps there are other solutions as well (some might even be non technical) but since we have the immediate problem now with something like 5000 to 10000 of the bogus messages per day it must be fixed sooner than later.
I just have to mention that I used the Pipeline Component Wizard available at codeplex to easily generate the component. Customizing it was very easy - literally just setting a variable that should be returned, to null.
The only relevant code to show here is this:
public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
{
inmsg = null; //Just throw it away...
return inmsg;
}
A while ago I wrote about getting filenames from suspended entries for support people that used a more simple routine to retrieve the details from the context field. I recently had to try to find some more details about non File adapter type of messages where the number of potential messages arriving was so high that using the normal Admin tool would just be a pain. So I embarked to retry figuring out how the encoding works of the imgContext field.The result is that I now can display it in a readable format and in the process I discovered how handy it could be to see what BizTalk is doing behind the scenes (well sort of). Of course, this is probably not supported by Microsoft to poke around the BizTalk databases but what do you expect from developers ;) (we like to poke, break and create things - including trouble for ourselves). The point of this idea is to read stuff from the database and in no way to try and update and mess around with the working of the product (BizTalk).
It turns out my previous attempts to read the encoded text failed because I was expecting the format to be too complex. Using the built in Base64 decoding functions of the .Net framework did not give me a valid output (probably because the input format was different from the standard it expects). So I wrote my own custom decoding function which is probably not so optimized or well written as the MS base libraries but, it works for me for single messages at a time. It does not check for any fancy things like different encodings or stuff and probably would only work for standard US English.
Thus here is the code sample:
private string EncodeText(string inputText)
{
string output = string.Empty;
try
{
if (inputText.EndsWith("\r\n") )
inputText = inputText.TrimEnd('\r', '\n');
if (inputText.StartsWith("0x"))
inputText = inputText.Substring(2);
byte[] bts = new byte[inputText.Length / 2];
for (int i = startoffset; i < inputText.Length - startoffset; i = i + 2)
{
bts[i / 2] = Convert.ToByte(inputText.Substring(i, 2), 16);
}
output = Bytes2String(bts);
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString(), "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
}
return output;
}
private string Bytes2String(byte[] bytes)
{
StringBuilder sb = new StringBuilder();
for (int i = startoffset; i < bytes.Length / 2; i++)
{
try
{
if ((bytes[i * 2] == 0) && (bytes[i * 2 + 1] == 0))
sb.Append(" ");
else if ((bytes[i * 2] > 0) && (bytes[i * 2 + 1] == 0))
sb.Append(char.ConvertFromUtf32(bytes[i * 2]));
else
sb.Append(char.ConvertFromUtf32(bytes[i * 2] * 256 + bytes[i * 2 + 1]));
}
catch (ArgumentOutOfRangeException)
{
//ignoring this for now...
}
}
return sb.ToString().Replace("\0", "");
}
The functions are very generic and can thus be used in other ways as well. I've attached the sample windows application that can be used to copy and paste the encoded text from a sql query result and then decoded. It is compiled with VS2008/ .Net 3.5 but might as well have been .Net 2.0. To use it simply run a query like 'select imgContext from Spool' on the BizTalkMsgBoxDb and copy the value of one of the rows. Paste it then in the utility's first text box and click one of the buttons. The input text usually starts with something like '0xD4E0906C1849...'.
Have fun and remember, do it all at your own risk...of finding out stuff.
One of the real limitations of BizTalk server (design issue) is the lack of details for when things go wrong - and lets face it there will always be things outside the control of the product, not even mentioning bugs in the product itself. For example, when dealing with plain files (flat files or xml) something that is critical to some people gets lost - the original filename. It is possible to retrieve it through the context of the message inside BizTalk but it has to be done one by one per message. This piece of information is not logged in the eventlog - which is really weird! Sometimes files fail in batches and then it becomes a real plain to retrieve filenames one by one. For that reason I created a simple tool to retrieve the filenames of suspended files in bulk.
The solution is a bit of a hack as it requires direct access to the BizTalk messagebox database. The tricky bit is to retrieve the filename from the message context as it is encoded in some propriety binary format. Through trial and error I decoded this context format and is now able to retrieve some of the properties inside it. Of course, there is no guarentee that MS will not change the format in the future.
Since suspended entries are stored by Host it is tricky to simply have one select statement to gather all suspended files at once. Therefore it has to be done host by host. This means at least 2 select statements have to be executed to retrieve the data.
The first step to retrieve the suspended queue (table) names are easy:
private StringCollection GetHostInstanceSuspendedQs()
{
string sql = "select name from sysobjects where name like '%[_]Suspended'";
StringCollection queues = new StringCollection();
DataSet ds = new DataSet();
using (SqlConnection conn = new SqlConnection(connectionString))
{
conn.Open();
using (SqlCommand cmnd = new SqlCommand(sql, conn))
{
SqlDataReader dr = cmnd.ExecuteReader();
while (dr.Read())
{
string queue = dr[0].ToString();
queues.Add(queue);
}
}
}
return queues;
}
Once you have a list of the suspended queues you can query the rest of the tables including the suspended queue in question:
private List<SuspendedEntry> GetSuspendedInstances(string hostInstanceSuspendedQ)
{
List<SuspendedEntry> list = new List<SuspendedEntry>();
string hostName = hostInstanceSuspendedQ.Substring(0, hostInstanceSuspendedQ.Length - 11);
string sql = "select b.nvcAdditionalInfo, s.imgContext, b.dtLastTouched, b.uidInstanceID, b.uidMessageID, " +
"s.PublishingServer, i.nvcErrorID, m.nvcName as [App], '%2' as [Host] " +
" from %1 b inner Join Spool s on b.uidMessageID = s.[uidMessageID ] inner join InstancesSuspended i on b.uidInstanceID = i.uidInstanceID inner join Services se on se.uidServiceID = b.uidServiceID inner join Modules m on m.nModuleID = se.nModuleID" +
" where (i.nState = 4) AND (i.nvcAdapter = 'FILE')";
sql = sql.Replace("%1", hostInstanceSuspendedQ);
sql = sql.Replace("%2", hostName);
DataSet ds = new DataSet();
using (SqlConnection conn = new SqlConnection(connectionString))
{
conn.Open();
using (SqlCommand cmnd = new SqlCommand(sql, conn))
{
cmnd.CommandTimeout = 300;
using (SqlDataAdapter da = new SqlDataAdapter(cmnd))
da.Fill(ds);
}
}
foreach (DataRow row in ds.Tables[0].Rows)
{
SuspendedEntry entry = new SuspendedEntry();
entry.InstanceId = row["uidInstanceID"].ToString();
entry.MessageId = row["uidMessageID"].ToString();
entry.LastTouchDate = DateTime.Parse(row["dtLastTouched"].ToString());
byte[] bytes = (byte[])row["imgContext"];
int receiveFileNameStart = SearchBytes(bytes, String2Bytes("ReceivedFileName"));
receiveFileNameStart = receiveFileNameStart + 41;
int receiveFileNameLen = (bytes[receiveFileNameStart] * 256) + bytes[receiveFileNameStart + 1];
byte[] fileNameBytes = CopyBytes(bytes, receiveFileNameStart + 4, receiveFileNameLen);
entry.FileName = Bytes2String(fileNameBytes);
entry.AdditionalInfo = row["nvcAdditionalInfo"].ToString();
entry.PublishingServer = row["PublishingServer"].ToString();
entry.ErrorID = row["nvcErrorID"].ToString();
entry.AppName = row["App"].ToString();
entry.Host = row["Host"].ToString();
list.Add(entry);
}
return list;
}
There are a couple of helper functions to do byte and string conversions. I'll include them in the attached zip file.
The result of all this is a nice list you can use to display in a tool or on a web page. This works only for files, a.k.a. messages that was received through the file adapter.
The ideal would have been that the filename was logged in the event log as soon as it was suspended. This is a crucial bit of information that is usually needed by the people monitoring the whole system (including BizTalk).
Due to multiple environments (BizTalk) I have to manage and also oversee deployments over these environments I have created a tool to compare the GACs (Global Assembly Cache) of machines. The first tool I created was a simple command line one and was quite limited (and a bit flawed oops). Then recently we started using 64bit machines and I realized the old utility does not work so good anymore. Thus I created a new utility (Win form based) that take inconsideration things like 32bit/64bit as well.
First some background on how this utility works:
This tool only list the difference between the GACs of the machines. No changes are made - That would be a very interesting and perhaps hairy experience!
To get the information of what is installed in the GAC is really simple. As long as you have local admin rights on the machine you can do a plain old command line 'dir' command on \\machine\c$\windows\assembly and its sub directories. There are multiple sub directories like GAC, GAC_MSIL etc.
1. Under each of these there are sub directories that list the assembly names (not the assembly itself)
2. Under these there is another sub directory that reports the version, culture and public token key values (name of directory)
3. Under these the actual dlls are stored.
Then it is simply a matter to list the directories and processing the names and put them into a list. Once you have the lists you can compare them to each other and get the differences.
Just one interesting bit. The plain GAC directory was the default for .Net 1.x. The GAC_MSIL is the default for .Net 2.0. There are also GAC_32 and GAC_64 ones that contain 32bit and 64bit assemblies - of course GAC_64 only applies to 64bit machines.
I'm not listing the entire source code here but here is the main function that does the bulk of the work.
private void GetGACFiles(string rootDirectory)
{
if (Directory.Exists(rootDirectory))
{
string[] assemblyDirectories = Directory.GetDirectories(rootDirectory);
foreach (string assemblyDirectory in assemblyDirectories)
{
string[] versionTokenDirectories = Directory.GetDirectories(assemblyDirectory);
foreach (string versionTokenDirectory in versionTokenDirectories)
{
string[] assemblyFiles = Directory.GetFiles(versionTokenDirectory, "*.dll"); //There should only be one... Don't loose your head
foreach (string assemblyFile in assemblyFiles) //sometimes there are multiple files
{
if (gacEntry.AssemblyPath.ToLower().Contains(gacEntry.AssemblyName.ToLower() + ".dll"))
{
GACEntry gacEntry = new GACEntry();
gacEntry.AssemblyName = Path.GetFileName(assemblyDirectory);
gacEntry.AssemblyPath = assemblyFile;
gacEntry.AssemblyVersion = GetVersionFromVersionToken(versionTokenDirectory);
gacEntry.Culture = GetCultureFromVersionToken(versionTokenDirectory);
gacEntry.PublicKeyToken = GetTokenFromVersionToken(versionTokenDirectory);
if (!assemblies.Contains(gacEntry)) //avoid duplicates
assemblies.Add(gacEntry);
}
}
}
}
}
}
There you have it. Not rocket science. It is provided as is...
Following on yesterdays post about creating a host the next step is to create a host instance. This sample create the instance and start it as well.
Here is the code:
---------------------------------------------------------------
using System;
using System.Management;
namespace CreateStartHostInstance
{
class AddStartHostInstance
{
/// <summary>
/// The main entry point for the application.
/// </summary>
[STAThread]
static void Main(string[] args)
{
string bts_WMINameSpace;
string bts_ServerAppTypeNameSpace;
string bts_HostInstanceNameSpace;
string user;
string pwd;
string hostName;
string serverName;
bts_ServerAppTypeNameSpace = "MSBTS_ServerHost";
bts_HostInstanceNameSpace = "MSBTS_HostInstance";
// WMI NameSpace for BizTalk Server
bts_WMINameSpace = @"root\MicrosoftBizTalkServer";
try
{
if (args.Length < 4)
{
ShowHelp();
}
else if (args[0] == "-?" || args[0] == "/?" || args[0] == "/h" || args[0] == "/h")
{
ShowHelp();
}
else
{
hostName = args[0];
serverName = args[1];
user = args[2];
pwd = args[3];
PutOptions options = new PutOptions();
options.Type = PutType.CreateOnly;
ManagementObject bts_AdminObjectServerHost = null;
ManagementObject bts_AdminObjectHostInstance = null;
System.Management.ObjectGetOptions bts_objOptions = new ObjectGetOptions();
// Creating instance of BizTalk Host.
ManagementClass bts_AdminObjClassServerHost = new ManagementClass(bts_WMINameSpace, bts_ServerAppTypeNameSpace, bts_objOptions);
bts_AdminObjectServerHost = bts_AdminObjClassServerHost.CreateInstance();
// Make sure to put correct Server Name,username and // password
bts_AdminObjectServerHost["ServerName"] = serverName;
bts_AdminObjectServerHost["HostName"] = hostName;
bts_AdminObjectServerHost.InvokeMethod("Map", null);
ManagementClass bts_AdminObjClassHostInstance = new ManagementClass(bts_WMINameSpace, bts_HostInstanceNameSpace, bts_objOptions);
bts_AdminObjectHostInstance = bts_AdminObjClassHostInstance.CreateInstance();
// Make Sure you correct HostName and MachineName for HostInstance name,
bts_AdminObjectHostInstance["Name"] = "Microsoft BizTalk Server " + hostName + " " + serverName;
object[] objparams = new object[3];
objparams[0] = user;
objparams[1] = pwd;
objparams[2] = true;
bts_AdminObjectHostInstance.InvokeMethod("Install", objparams);
Console.WriteLine("Host instance created successfully!");
bts_AdminObjectHostInstance.InvokeMethod("Start", null);
Console.WriteLine("Host instance started successfully!");
}
}
catch (Exception e)
{
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine(e.Message);
Console.ForegroundColor = ConsoleColor.White;
}
}
private static void ShowHelp()
{
Console.WriteLine("Usage: CreateStartHostInstance.exe <HostName> <ServerName> <UserName> <Password>");
}
}
}
---------------------------------------------------------------
Something I only discovered this morning (thanks Brandon) is a nice small utility to stop/start BizTalk 2006 applications. You would have thought MS should have include this functionality in the plain BTSTask utility.
It can be found here
The photo bug has bitten me and I'm taking pictures whenever I can. Also, it helps to now have a proper SLR camera...
Check out http://www.flickr.com/photos/rudolfhenning/
Many times you hear people saying went to a dark place and they could see millions of stars... Well, it turns out that it is not really possible to see that many even if you had the eye sight of the six million dollar man....
In reality you can at most see around 2500 stars even in the best of conditions - aka perfectly dark sky. The total number of stars visible to the human eye on Earth is around the 8500. Of course you can only see one half the sky at any one time (unless you can see through the planet) plus there are various atmospheric reasons that limit the number of stars you can see near the horizon.
If you want to read a bit more detail about it check this out
Apologies if this is not entirely on topic for this site but I'm trying to spread the message and its influence as wide as possible.
Hopefully most of you have heard about this story already but in case you haven't..
Over a week ago a little 7 year old was abducted from her home and she has been missing since then. The family has started a campaign to raise awareness about this case and also about the general safety of all children in our country. In the broader sense it is message to show how many dangers there are that threaten our children but also ourselves. In the end the government must take notice and do something (more than what they are doing now).
Several web sites, including the news sites like news24 and iAfrica.com have regular reports about the story.
Another web site (http://www.eblockwatch.co.za) has a live forum going with updates as news become available or things happen.
Sheldean is still missing and the police is doing all they can to help. The hope is that it would become so difficult for criminals to do these kind of things that it won't be worth their while doing this again. Of course, we live in a non perfect world but we have to keep on trying.
More Posts
Next page »