Symbian Emulator Ethernet Connectivity
[This is a copy of a post I posted to an internal blog for members of our early code access program at Red Five Labs. I'm reposting here as it might be beneficial for other people having problems with the S60 FP2 emulator ethernet connectivity on Windows Vista or on XP/2003 machines with Hyper Threaded CPUs ]
Earlier versions of the Series 60 (now S60) SDK's, particularly the S60 2nd Edition FP2 SDK, had some quirks when it came to connecting to the internet using the default Ethernet connectivity drivers.
This post addresses resolving the issues listed below on the S60 2nd Edition FP2 SDK . The two biggest problems users face when trying to get ethernet connectivity working with the emulator are the following :
- The emulator would cause a blue screen on machines with Hyper-Threaded CPU's. This typically happens a short while after the initial connection was established and affects both Windows XP and Windows Server 2003 machines. The faulting driver would be listed as npacket.sys which is the default packet driver used by the emulator. Some threads and a little more information on these issues available here and here.
- On Windows Vista initiating an ethernet connection from the emulator leaves with a KErrServerTerminated error (-15 Server has closed) rendering networking on the emulator on Windows Vista useless.
While the problems are not neccessarily related both can be resolved by following the steps set out in this post.
Some background on troubleshooting the connectivity problems : The Symbian emulator, when running the debug build, has very verbose output which can be viewed by attaching a debugger like Visual Studio to the epoc.exe process or viewing debug output using a tool like DebugView (Microsoft/Sysinternals).
Viewing the debug output on Windows Vista will reveal the following debug statements prior the leave happening :
pcap_open_live() failed to open -> The operation completed successfully.
CloseAdapter
CloseAdapter() File not Open
The function pcap_open_live is a function usually exported from WinPcap, the Windows Packet Capture Library. WinPcap is an open-source library typically used for link-layer network access in Windows environments, allowing applications to capture and transmit packets without going through the Windows protocol stack. The Symbian emulator uses this library, either directly or through the npacket.sys derivative as we'll see later, to emulate a new device on the network allowing it to receive and transmit packets as if it were physically connected to the network.
To understand where this fits into the ethernet connectivity implementation on the emulator the next section will give a quick overview of how Symbian device drivers are typically implemented...
Overview of Symbian Device Drivers
Symbian device drivers are usually split into two distinct parts which is called the LPP-PDD model :
- The Logical Device Driver - the LDD
- The Physical Device Driver - the PDD
The interface to a device driver is typically the same for a number of devices but the hardware implementation is different thus the bulk of the device driver is contained in the LDD with the platform or hardware specific bits contained in the PDD layer.
This clean seperation makes it easy to port the driver to new hardware and also allows the physical implementation to be swapped out. For more information on the Symbian OS Device Driver model see the Symbian Developer Library.
For our purposes it's enough to know that the ethernet device driver implementation for the emulator complies to this same model :
- The LDD contained in ETHERCARD.LDD
- The PDD configurable but by default contained in ETHERWINS.PDD
These settings can be changed by manually editing the CommDB database and modifying the [LANBearer] section but that is beyond the scope of this post.
These files are located in the relevant system folder. ie. <EPOC32>\release\wins\udeb for the debug emulator. The emulator build typically used while developing.
Bring on the Substitute
Having a look at the module dependencies of the default etherwins.pdd driver with the Dependency Walker tool, included with Visual Studio, we see that a dependency exists to npacket.sys. It seems that npacket.sys is a problematic port of the winpap library and where most of the ethernet issues originate from. As this implementation don't allow us to do much about the problem other avenues would need to be pursued.
Looking back at a previous installation of the Symbian OS SDK for OS 7.0 (2nd Edition) a seperate download were available from Symbian Developer Network to enable ethernet connectivity in the emulator. The physical ethernet device driver component that shipped with this update has a dependency on WPCAP.DLL which is the standard high-level winPcap library, the interface into the low-level packet.dll library which forms the core of winPcap. Using this version would be ideal as we would like to have the ability to use the latest WinPcap version and be able to troubleshoot issues with a known library instead of relying on the black box npacket.sys.
Symbian have been doing an exceptionally good job of maintaining compatibility between different releases (up to the binary break in OS 9) and this is no exception. There's been no break in the interface so the older etherwins.pdd file can be used as a drop-in replacement for it's newer cousin.
Switching to this implementation will give us control over the lowest level of the emulator's ethernet connectivity implementation as the source for WinPcap is freely available allowing us to debug and troubleshoot the problems.
- Download and install WinPCap 4.0 Beta 2 (The latest version at the time of this post) Version 4.2 includes support for Vista. [Optional, DIY : Download and build the source]
- Download the "Ethernet Connection for the Symbian OS Emulator" package from Symbian Developer Network. [Optionally also download the instructions as it contains more information on the implementation]. The download is available from here. Scroll down the the Emulator/Simulator section for the actual download links.
- Following the steps in the setup guide, unzip the package containing the ethernet driver files over your existing Symbian OS SDK installation. The following files are replaced or added during this process and should be backed up should you want to roll-back to the default implementation at some stage :
epoc32\release\wins\udeb\ETHERWINS.PDD
epoc32\release\wins\udeb\urel\ETHERWINS.PDD
epoc32\tools\configchange.pl
epoc32\tools\netcards.exe
epoc32\wins\c\7.0sDevKit_ethernetced.cfg
epoc32\wins\c\series60_2.1_ethernetced.cfg
epoc32\wins\c\v7.0_ethernetced.cfg
epoc32\winscw\c\7.0sDevKit_ethernetced.cfg
epoc32\winscw\c\series60_2.1_ethernetced.cfg
epoc32\winscw\c\v7.0_ethernetced.cfg
- Next the networking needs to be configured, the procedure for this is slightly different than in the original setup. (Again detailed steps available in the documentation that accompanies the install)
- [Optional] Under epoc32\wins\c rename the file series60_2.1_ethernetced.cfg to ethernetced.cfg. This file contains a dumped CommDB with ethernet configured for using DHCP. Import this file into the CommDB using the ced.exe utility that ship with the Symbian SDK. This step is entirely optional and actually just included here for compliance to the original docs. With the default settings the CommDB is actually already configured to use the same drivers, with the same names located in the same locations.
- Next some configuration of the network card device id and fake MAC address to be used by the emulator needs to be configured. The following two files are required for this configuration (The paths given are relative to the emulator's file system, ie. c maps to epoc32\wins\c on Windows) :
c\system\data\etherdriver.dat
c\system\data\ethermac.dat
- In a default installation of the OS 8.0 SDK these files are located under epoc32\data. The older etherwins.pdd expects to find these files under epoc32\wins\c\System\data. Move the files to the expected location.
- Configure the device id for the physical ethernet network adapter installed on the machine to be used by the emulator and WinPcap. I've found that the easiest way to obtain these network device id's is to use other WinPcap dependent tools like Ethereal. In Ethereal these device id's can be found under the adapter selection screens. Alternatively you can make use of the WinPCap libraries yourself to query for available adapters. (Ethereal is however useful as it also offers an easy way to configure the kernel mode NPF (Netgroup Packet Filder) service to start when the system boots which allows the emulator to access the network without elevated priviledges on Windows Vista or a non-administrative user on Windows XP)
The device id identifying the adapter to use should have a format similar to the sample below :
\Device\NPF_{F78681D1-F718-4FF2-ACFD-FCDCB456789A}
In the Nokia WinPcap version that ships with the SDK this device ID is formatted slightly different. If you open the original etherdriver.dat file you'll notice that the NPF_ prefix is missing. This is also partly the reason why the netcards.exe utility that is normally used by the Symbian SDK to obtain the device id is not useful for the substituted driver. When adding the device id to the etherdriver.dat file, ensure that there is not extra white-space or additional characters present as an exact match is required by WinPcap to bind to the correct adapter. (I've had a newline character accidently added to my device id which caused endless headaches and several hours of painful debugging to resolve) If after all the steps were followed and the PDD replaced the pcap_open_live call still fails to open, this can be seen in the debug output, this would be a good place to start looking. - Configure the fake MAC address to be used by our emulator. This needs to be entered in the ethermac.dat file. A good guideline to follow is to use your physical network adapters MAC address and modify the first byte to be 0x02 to be "locally unique. Your adapters MAC can be obtained by running ipconfig /all from a Windows command prompt. The format for this entry is shown below:
021486a790cf
With these modifications in place your emulator should be configured and ready to use the older version of the PDD component of the Ethernet Device driver and the latest WinPcap build, allowing you to connect the emulator to the internet through your host machine's ethernet connection.
On Windows XP and Windows Server 2003 this updated configuration should resolve the bluescreen and stop issues that hamper development which requires a network connection on Hyper Threaded machines.
This also allows the ethernet connectivity to function without problems on Windows Vista.
Some Notes
- When running a self-compiled version of WinPcap you might run into a missing NPPTools.dll dependency on Vista. This library no longer ships on Vista and a quick workaround for this is to compile the packet.dll project in NT4 configuration which doesn't have this dependency.
- When installing Ethereal you can optionally select to start the NPF service at startup which allows non-administrative users or processes running without elevated priviledges to connect to the internet using the ethernet connection on the host machine.
Links