Including WCF services as part of the MSI?

Topics: IIS and Web Services
Sep 23, 2010 at 3:40 PM
Edited Sep 23, 2010 at 5:17 PM

As the documentation is fairly sparse, what are the exact steps necessary to get BTDF to successfully install WCF services created from orchestrations? I'm using the BT WCF Service Publishing Wizard after I build and before I deploy. BizTalk Admin Console automatically picks up all the files in the virtual directory when I specify that vdir as a resource, but the Framework only creates and configures the vdir and ignores all the files within.

Edit:

After looking through the output from the BuildDebugMsi batch file, it appears there is a problem with the depth of directories:

 Copying file from "..\..\..\..\..\inetpub\wwwroot\ESB\App_Data\ESB.schemas.MasterAgents.xsd" to "obj\Debug\redist\Vdirs\..\..\..\..\..\inetpub\wwwroot\ESB \App_Data\ESB.schemas.MasterAgents.xsd".

 

Obviously my project is pretty deep in the folder structure. However, the framework shouldn't be copying files to "obj\Debug\redist\Vdirs\..\..\..\..\..\inetpub\wwwroot\ESB \App_Data\ESB.schemas.MasterAgents.xsd" but rather to "obj\Debug\redist\Vdirs\inetpub\wwwroot\ESB \App_Data\ESB.schemas.MasterAgents.xsd" - is there a fix for this? Moving the directory depth of my project is not an option.

Coordinator
Sep 23, 2010 at 6:43 PM

Typically I export the service and then move the exported project folder from inetpub\wwwroot into my BizTalk solution folder structure, and add the project into my solution.  It ends up sitting next to my Orchestrations and other projects.  When it's installed on the server, the vdir points to something like \Program Files\My BizTalk Project\Vdirs\MyWCFService and your vdir physdir path in the .BTDFPROJ is just ..\MyWCFService.  If I ever export the WCF service again, I use WinMerge to compare the two folders and selectively decide which changes to bring in.

Does that help?

Thanks,
Tom

Sep 23, 2010 at 7:43 PM

We're in a multi-developer situation with continuous integration so that's probably well outside our convenience zone. It would be helpful in a further version if the VDirList structure looked more like:

<VDirList Include="*">
 <VDir>ESB</VDir>
 <SourcePath>c:\inetpub\wwwroot\ESB</SourcePath>
 <DestinationPath>c:\inetpub\wwwroot\ESB</SourcePath>
 <AppPool>DefaultAppPool</AppPool>
</VDirList>

I've looked through the BuildDebugMSI output and it is most certainly copying the files as stated (which leads me to think that the files are in the MSI?) but when I install the MSI on another machine, the files never get copied over.

Thanks for the suggestion, we'll discuss it among our team and see if it's something we want to pursue.

Sep 23, 2010 at 9:08 PM

Did you set this to true?

    <IncludeVirtualDirectories>true</IncludeVirtualDirectories>

Neal Walters

 

Sep 23, 2010 at 9:12 PM

Yeah Neal. I looked through the output for the MSI build and it is definitely processing the files in the virtual directory. They just don't get copied in when I install the MSI on another machine. I'm locally testing Tom's suggestion in addition to remapping my virtual directory to the solution folder to see if that will be workable.

Sep 23, 2010 at 9:18 PM
Edited Sep 23, 2010 at 9:19 PM

What version of IIS?  Seems like with IIS7 we had to install the IIS6 tools or compatibility features.
Ok, that's the only possible quick fix I knew of.  I used this feature at previous client, and it worked nicely.

Is it creating the IIS application and app pool (for test purposes might want to put a non-default for the app pool)?

Neal

Sep 23, 2010 at 9:32 PM

Yes, it creates and configures the destination virtual directory and app pool appropriately, it just wasn't copying the files.

But I've fixed that now:

1) I created a folder in the same path as the \Deployment\ folder. Also in that path is my solution file and a folder containing my other Biztalk files (Orchestrations, schemas, transforms, etc). I now have something like c:\TFS\branches\cycle 3\Integration\VDir

2) Remapped my local virtual directory to point to this folder.

3) Ran the BT WCF Publishing tool to create my service fresh in this new folder.

4) BuildDebugMSI

5) Uninstalled previous application, removed application via Biztalk Admin Console, wiped my virtual directory clean.

6) Ran the MSI on my local development instance, installing to c:\program files\esb for biztalk\1.0\

 

The end result is that in the \1.0\ folder there is now a folder for my virtual directory. The MSI re-maps my virtual directory to this new folder. All the files are installed as expected. The next time I want to re-publish the services all I need to do is remap the virtual directory to the existing folder I created in Step #1. Running the MSI will change it back. Not sure how TFS will respond to adding that folder to source control when it comes to future edits. I may not add the files at all, only the folder itself.

Coordinator
Sep 23, 2010 at 9:43 PM

Yep, that sounds pretty much like the model that I described.  I've used that arrangement very successfully in the past.  I have actually checked the WCF service files into source control, however, I had also created a custom MSBuild task that exported the schema files during a build.  That meant that I didn't need to check in the App_Data (?) folder where all the schemas were located.  They were just generated on the fly.

Thanks,
Tom

Feb 26, 2011 at 12:03 AM

Tom,

Can you explain how you do your schema export? Does it also generate the ServiceDescription.xml file?

Thank you.

Elias

Coordinator
Feb 28, 2011 at 6:21 AM

Yes, it uses BizTalk's own service export code.

Here's the code -- you can use it if you'd like.  This was created for MSBuild 2.0 but should work the same with 3.5 or 4.0.  Just follow the MSDN directions for creating a custom MSBuild task library.  Once I had the custom MSBuild task DLL in hand, I referenced it inside my WCF service project to do the export automatically (see below).

using System;
using System.Collections;
using System.Collections.Generic;
using System.Reflection;
using System.Text;
using Microsoft.Build.Framework;
using Microsoft.Build.Utilities;

namespace BuildTasks.BizTalk
{
    /// <summary>
    /// Written by Thomas F. Abraham (http://www.tfabraham.com)
    /// 
    /// This MSBuild task uses the BizTalk 2006 R2 WCF publishing libraries to export schemas
    /// and definition files for a WCF Web service exposed from BizTalk.
    /// 
    /// Published services contain copies of all of the schemas related to the service in a
    /// set of XSD files on disk.  This means that every time one of those schemas changes,
    /// the service must be re-published.  That's where this task comes in.  It removes the
    /// need to store copies of the published schema files because it can re-publish them
    /// on demand.
    /// 
    /// This task requires that the service be manually exported once with the WCF Service
    /// Publishing Wizard.  Among other files, the export process creates a definition file called
    /// WcfServiceDescription.xml, which captures the settings used in the wizard.  The file
    /// is saved to wwwroot\[service]\App_Data\Temp.  The task requires this definition
    /// file to reproduce the original export.
    /// 
    /// There are only four files that should be preserved in source control from the published
    /// service: [service].svc and web.config in wwwroot\[service] and the service description
    /// and binding XML files in wwwroot\[service]\App_Data\Temp.  Everything in App_Data can
    /// be recreated with this task.
    /// 
    /// This task does not directly reference the BizTalk assemblies.  The WcfPublishingAssemblyName
    /// property can override the .NET assembly name, which defaults to that of BizTalk 2006 R2.
    /// </summary>
    public class WcfServicePublisherTask : Task
    {
        private const string BizTalkWcfPublishingAssembly =
            "Microsoft.BizTalk.Adapter.Wcf.Publishing, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35";

        private string _outputPath;
        private string _serviceDescriptionPath;
        private string _wcfPublishingAssembly = BizTalkWcfPublishingAssembly;

        /// <summary>
        /// Path to the service home directory. App_Data will be created below this folder.
        /// </summary>
        [Required]
        public string OutputPath
        {
            get { return _outputPath; }
            set { _outputPath = value; }
        }

        /// <summary>
        /// Path to the WcfServiceDescription.xml file saved from a manual publication of the service.
        /// </summary>
        [Required]
        public string ServiceDescriptionPath
        {
            get { return _serviceDescriptionPath; }
            set { _serviceDescriptionPath = value; }
        }

        /// <summary>
        /// The complete assembly name (name, version, culture, public key) of the BizTalk
        /// WCF Publishing assembly. Default value is correct for BizTalk 2006 R2.
        /// </summary>
        public string WcfPublishingAssemblyName
        {
            get { return _wcfPublishingAssembly; }
            set { _wcfPublishingAssembly = value; }
        }

        public override bool Execute()
        {
            try
            {
                //
                // This code is based on Microsoft.BizTalk.Adapter.Wcf.Publishing.Publisher.Export().
                // Since most of the classes and methods are private, this code has to use reflection
                // to do most of the work.  It was developed and tested against the original release
                // of BizTalk Server 2006 R2.
                //

                // First, read in the service description file saved from a manual run of the
                // WCF Service Publishing Wizard.
                Type wsdType = Type.GetType(
                    "Microsoft.BizTalk.Adapter.Wcf.Publishing.Description.WcfServiceDescription, " + _wcfPublishingAssembly, true);
                object svcDescription =
                    wsdType.InvokeMember("LoadXml", BindingFlags.Static | BindingFlags.Public | BindingFlags.InvokeMethod, null, null, new object[] { _serviceDescriptionPath });

                Type wmtType = Type.GetType(
                    "Microsoft.BizTalk.Adapter.Wcf.Publishing.Description.WcfMessageType, " + _wcfPublishingAssembly, true);

                IDictionary uniqueMessageTypes = null;

                // Build a list of the message types defined in the service description file.
                Type publisherType = Type.GetType(
                    "Microsoft.BizTalk.Adapter.Wcf.Publishing.Publisher, " + _wcfPublishingAssembly, true);
                uniqueMessageTypes =
                    publisherType.InvokeMember("GetUniqueMessageTypes", BindingFlags.Static | BindingFlags.NonPublic | BindingFlags.InvokeMethod, null, null, new object[] { svcDescription }) as IDictionary;

                // Create a new instance of the WebServiceImplementation class, which will carry
                // out the save to disk later on.
                Type wsiType = Type.GetType(
                    "Microsoft.BizTalk.Adapter.Wcf.Publishing.Implementation.WebServiceImplementation, " + _wcfPublishingAssembly, true);
                object wsi = wsiType.InvokeMember("", BindingFlags.CreateInstance, null, null, null);

                // Create a BtsServiceDescription object based on the service description.
                Type bsdeType = Type.GetType(
                    "Microsoft.BizTalk.Adapter.Wcf.Publishing.Exporters.BtsServiceDescriptionExporter, " + _wcfPublishingAssembly, true);
                object bsd =
                    bsdeType.InvokeMember("Export", BindingFlags.Static | BindingFlags.Public | BindingFlags.InvokeMethod, null, null, new object[] { svcDescription });

                // Set the BtsServiceDescription object into the WebServiceImplementation object's
                // BtsServiceDescription property.
                wsiType.InvokeMember(
                    "BtsServiceDescription", BindingFlags.SetProperty | BindingFlags.Instance | BindingFlags.Public, null, wsi, new object[] { bsd });

                // Add the WCF-default DataContractSerializer schema to the WebServiceImplementation object.
                MethodInfo asstiMethod =
                    publisherType.GetMethod("AddSerializationSchemaToImplementation", BindingFlags.Static | BindingFlags.NonPublic);
                asstiMethod.Invoke(null, new object[] { wsi });

                MethodInfo pwmtMethod =
                    publisherType.GetMethod("ProcessWcfMessageType", BindingFlags.Static | BindingFlags.NonPublic);

                Log.LogMessage(MessageImportance.Normal, "Exporting WCF service artifacts from {0}...", _serviceDescriptionPath);

                // For each unique message type defined in the service description, extract the schemas
                // into the WebServiceImplementation object.
                foreach (object type in uniqueMessageTypes.Values)
                {
                    pwmtMethod.Invoke(null, new object[] { wsi, type });
                }

                // Now that we have everything we need in the WebServiceImplementation object,
                // it can save everything out to disk.
                MethodInfo stfMethod =
                    wsiType.GetMethod("SaveToFolder", BindingFlags.Instance | BindingFlags.Public);
                stfMethod.Invoke(wsi, new object[] { _outputPath });

            }
            catch (Exception ex)
            {
                Log.LogErrorFromException(ex);    
                return false;
            }

            return true;
        }
    }
}

This was inside the .csproj file for my WCF services (and the WCF project was located with and included in my BizTalk solution):

<UsingTask TaskName="BuildTasks.BizTalk.WcfServicePublisherTask" AssemblyFile="..\ExternalDependencies\BuildTasks.BizTalk.dll" />
...
<Target Name="AfterBuild">
  <WcfServicePublisherTask ServiceDescriptionPath="Services\v1\Definition\WcfServiceDescription.xml" OutputPath="Services\v1" />
</Target>

I hope this helps.  It worked great for us.

Thanks,
Tom

Mar 2, 2011 at 1:58 PM

Thank you Tom!

From my understanding these are the steps you suggest, please correct me if I'm wrong:

-Add a WCF project (or this can be also a simple Web project?) to the Biztalk solution for every endpoint that you want to publish.

These projects should be configured to use IIS vdirs instead of Visual Studio Development Server.

-Use Biztalk WCF Publishing Wizard to publish the WCF endpoint using the vdirs of the previously created WCF projects.

-Remove all files except the svc file, web.config and App_Data\Temp\WcfServiceDescription.xml.  The WcfServiceDescription.xml file can be moved to the root of the project.

-Add the custom task to the project afterbuild, to autogenerate the files in the App_Data and App_Data\Temp folders using WcfServiceDescription.xml as input.

Thank you.

elias

Coordinator
Mar 3, 2011 at 6:03 PM

Hi Elias,

That's pretty much correct.  The project could be a Web or WCF project.  At the time, I first used the BizTalk Service Publishing Wizard to create the WCF project.  It usually ended up in c:\inetpub\wwwroot\<service>, so I then physically moved that folder into my BizTalk solution's folder structure (from what I recall a couple of years ago).  I believe I even had two endpoints defined in the same WCF project.  I guess it just depends if you are letting the BizTalk wizard generate the WCF project (.csproj, web.config, etc.) for you, or if you prefer to create it yourself.

Let me know if you have any trouble getting it going.

Thanks,
Tom

Mar 10, 2011 at 10:57 PM

Hi Tom,

I followed the above approach but getting an error. "Error    5    Exception has been thrown by the target of an invocation." at WcfTask Line.

Steps

Published Schema as WS

Created WS project(A) under the main solution

moved web.config, .asmx and webservicedescription.xml from wwwroot to WS project A.

moved webservicedescription.xml to "Services\v1\Definition\" folder, I tried at project level also just by giving path="webservicedescription.xml"

build the project.

 

Thanks

Mar 10, 2011 at 11:15 PM

Hi Tom,

I was doing some research, and it looks like the above code works only if schema is published as WCF service, any idea what needs to be done if schema is published as Webservice?

 

Thanks

Coordinator
Mar 10, 2011 at 11:29 PM

I assume you mean as ASMX?  "Webservice" could apply to either ASMX or WCF.  Yes, this was designed for WCF, so I'm not sure how it would have to be modified for ASMX.  It could be that you could create a similar custom MSBuild task but pointing to an ASMX service publisher.

For what it's worth, anything you can do with ASMX you can also do with WCF (but better).  Personally, I no longer use ASMX at all.  You would use basicHttpBinding in WCF to create a plain, simple SOAP web service very similar to ASMX.  If you need to be very compatible with ASMX, there are documented settings in WCF that will give you that compatibility.  You just need .NET 3.0 or newer on your server(s).

Thanks,
Tom

Mar 22, 2011 at 3:18 PM

Tom,

It works great!

Have you considered adding the task to the Deployment Framework tasks?

Coordinator
Mar 22, 2011 at 3:55 PM

Maybe I should do that!  I wrote it a few years ago and then didn't know where to put it for sharing.  The Deployment Framework can be kind of like a treasure hunt...  It can probably do a number of things that most users have never realized.  One non-obvious thing is some of the helper MSBuild tasks that are already included and available for use in the .btdfproj file.  Maybe I could include it as a standalone task assembly instead of bundling into the existing assembly.

Thanks,
Tom

Apr 14, 2011 at 11:51 PM

I think it would be awesome to have documentation on all the custom tasks BTDF defines!

 

Coordinator
Apr 15, 2011 at 12:16 AM

Documentation always ends up last..  I'd love to have a technical writer to create some great, robust documentation and examples.  It's tough to find the time to keep up with support and enhancements, but I will do my best to enhance the documentation over time!

Thanks,
Tom

Aug 23, 2011 at 8:53 PM

I would like to setup BTDF for seperate WCF Service, which is not part of the BizTalk Solution.

Could you please advise on where i can use below tag in btdf file, so that btdf script wont assume the soultion as BizTalk solution.

  <DeployBizTalkMgmtDB>false</DeployBizTalkMgmtDB>

 I would like to use this below tag to deploy my WCF Service. This is the only tag set to be true to deploy webservice.

<

 

IncludeCompsAndVDirsAsResources>True</IncludeCompsAndVDirsAsResources

>


Coordinator
Aug 24, 2011 at 6:30 AM

Are you saying that you'd like to use the Deployment Framework to deploy a web service that has nothing to do with BizTalk?  If so, I can't offer support for that usage.

DeployBizTalkMgmtDb is an MSBuild property, so it can go in any PropertyGroup.

Thanks,
Tom

Nov 7, 2012 at 10:53 PM

Hi Tom,

I have a scenario to extend the BTDF for web services and copy some settings files from Build Server as part of MSI generation to the BizTalk Server.

1 - My customer has published Orchestrations as a Web Services and they are hosted locally on inet default directory. Now duing the build using .btdf file they want to bundle (or copy) files from the www inet default directory and move to the BizTalk Server created virtual directory after MSI deployment. I was able to create the virtual directory successfully using the following VDirList group item .btdf file BUT all .asmx service files are not getting COPIED to the destination virtual directory. Please let me know what is the solution for to handle this scenario.....And my customer is not willing to move the web service file in the BizTlak solution....

<ItemGroup>
 <VDirList Include="*">
   <Vdir>MyWebService</Vdir>
   <Physdir>..\MyWebService</Physdir>
   <AppPool>MyAppPool</AppPool>
   <AppPoolNetVersion>v4.0</AppPoolNetVersion>
 </VDirList>
</ItemGroup>

2 - There is a user defined settings.xml file which is being used by the BizTalk Server at runtime. Customer wants to move that file in certain folder location on BizTalk (Destination) Server. I have used the external assemblies group item to bundle that file but unable to use the Copy in certain folder. I tried the following command but it didn't work. Followign copy target is not working once I put it into .btdf file. How should I use MSBuild Tasks in .btdf file to copy file during the deployment using BTDF MSI wizard on destination server.

<Target Name="CopyFiles">
        <Copy
            SourceFiles="@(MySourceFiles)"
            DestinationFolder="c:\MyProject\Destination" />

</Target>

Please tell me that how can I use MSBuild tasks in .btdf file . I am unable to reference those and it is not working.....

Immediate help is required. Thanks a lot for you input....

Regards,

ST

Coordinator
Nov 8, 2012 at 5:50 AM

1) If you cannot move the web service files into the BizTalk solution folder (which I assume means that they are also not in source control), then you'll have to assume that the web service files exist on the server that is used to build the BTDF MSI.  I'm assuming that is what you mean by "du[r]ing the build using .btdf file they want to bundle" the files.  In that case you will add <Target Name="CustomRedist"> to your .btdfproj and inside it you'll use a Copy task to copy the files from the inetpub location to $(RedistDir)\MyWebService.  The Advanced sample app demonstrates this kind of copy.

2) You should be using an AdditionalFiles ItemGroup to include settings.xml into the BTDF MSI.  Then add <Target Name="CustomDeployTarget">(reference) and inside it use a Copy task to copy from ..\settings.xml to the destination folder.

Thanks,
Tom