I’m currently working on moving a large legacy system from .NET Framework 4.8 to .NET 6. Since this is a large system the move will take years and we need to work iteratively meaning both systems will co-exist over a few years.

This means that integration tests we already have for the legacy system now needs to execute code over two application domains running two completely different CLRs. I ended up making a little service that the legacy code can call to execute code in the new system.

First we create a new web API project with a single controller.

[ApiController]
[Route("[controller]")]
public class TestController : ControllerBase
{
    private static readonly Dictionary<Guid, IServiceProvider> ServiceProviders = new();

    public bool Get()
    {
        return true;
    }

    [HttpPost("SetupTest")]
    public async Task SetupTest(Guid testId, string connectionString)
    {

        var collection = new ServiceCollection();

        var provider = collection
            .AddCqs(configure => collection.AddMassTransitTestHarness(cfg =>
            {
                cfg.UsingInMemory((ctx, mem) =>
                {
                    mem.ConfigureTestHarness(ctx);
                    mem.AddOutbox(ctx);
                    mem.ConfigureEndpoints(ctx);
                });
                configure(cfg);
            }))
            .AddDbContext<PcDbContext>(b =>
            {
                b.UseSqlServer(connectionString);
            })
            .AddBusinessCore()
            .AddRepositories()
            .AddDomain()
            .AddTestHarness()
            .AddTestGuidFileRepository()
            .BuildServiceProvider();


        var harness = provider.GetRequiredService<IHarness>();
        await harness.Start();

        ServiceProviders.Add(testId, provider);
    }

    [HttpPost("TeardownTest")]
    public void TeardownTest(Guid testId)
    {
        ServiceProviders.Remove(testId);
    }


    private static readonly JsonSerializerOptions Options = new() { PropertyNameCaseInsensitive = true };

    [HttpPost("ExecuteCommand")]
    public async Task ExecuteCommand(Guid testId, string cmdType)
    {
        var provider = ServiceProviders[testId];

        var cmd = (await JsonSerializer.DeserializeAsync(HttpContext.Request.Body, Type.GetType(cmdType)!, Options))!;
        await provider.GetRequiredService<IBus>().Publish(cmd);
        await provider.GetRequiredService<IHarness>().WaitForBus();
    }
}

First we just create a simple ping method that we can call to see if the service is up when setting up the test.

Then we have a SetupTest method that takes a Test ID and a SQL connection string, our domain is driven by a SQL server for persistence. This method configures the service collection close to what it would look like when running our system in Azure. Lastly it adds the IoC container to a lookup collection using the Test ID as key.

Finally we have a ExecuteCommand method that can execute commands sent from the legacy tests. It simply gets the IoC container from the lookup collection, deserialize the command and puts the command on our bus. We then wait for the system to digest the command and any messages that happens to be spawned by the command.

That’s it for the service. Now we need a way of starting the service from our legacy tests. Also on our build agents there can be multiple versions of the code running so we need to have multiple services up and running simultaneously.

protected async Task StartNetCoreTestRunner()
{
    if (_coreRunner == null)
    {
        #if !DEBUG
        _coreRunnerPort = GetFreePort();
        #endif

        Console.WriteLine($"Running CoolSystem on port: {_coreRunnerPort}");

        var project = "Customer.CoolSystem.LegacyTester";
        var path = TraverseUpAndStopAtRootOf(AppDomain.CurrentDomain.BaseDirectory, project).FullName;

        var cli = $"{Environment.ExpandEnvironmentVariables("%ProgramFiles%")}\\dotnet\\dotnet.exe";
        var targetPath = $@"{path}\{project}";
        var cliPathExists = File.Exists(cli);

        Console.WriteLine($"Path to cli: {cli}");
        Console.WriteLine($"Cli exists at path: {cliPathExists}");
        if (!cliPathExists) Console.WriteLine("Reverting to Environment Variable %dotnet%");

        _coreRunner = new Process
        {
            StartInfo = new ProcessStartInfo
            {
                FileName = cliPathExists ? cli : "dotnet",
                Arguments = $"run --property:Configuration=Release --configuration Release --urls \"http://localhost:{_coreRunnerPort}\"",
                UseShellExecute = false,
                RedirectStandardOutput = true,
                CreateNoWindow = true,
                WorkingDirectory = targetPath,
                RedirectStandardError = true
            }
        };
        _coreRunner.Start();

        System.Threading.Tasks.Task.Run(async () =>
        {
            while (!_coreRunner.HasExited && !_coreRunner.StandardOutput.EndOfStream)
            {
                await _coreRunner.StandardOutput.ReadLineAsync(); //MassTranssit doesnt work correctly if StandardOutput isnt consumed
            }
        });

        var ready = false;
        var stopwatch = new Stopwatch();
        stopwatch.Start();

        while (!ready)
        {
            try
            {
                await new HttpClient { Timeout = TimeSpan.FromSeconds(5) }.SendAsync(new HttpRequestMessage
                {
                    Method = HttpMethod.Get,
                    RequestUri = new Uri(GetCoreTestRunnerUrl())

                });

                ready = true;
            }
            catch (Exception e)
            {
                if (_coreRunner.HasExited)
                {
                    var error = _coreRunner.StandardError.ReadToEnd();
                    throw new Exception($"Cant ping CoolSystem and dotnet run exited with code: {_coreRunner.ExitCode} and error output: {error}");
                }

                if (stopwatch.Elapsed > TimeSpan.FromSeconds(30))
                {
                    Console.WriteLine($"Exception after timeout period: {e.Message}");
                    throw;
                }
            }

        }

    }

    var conn = _config["CoolSystemConnectionString"];
    _testId = Guid.NewGuid();
    var uri = new Uri($@"{GetCoreTestRunnerUrl()}SetupTest?testId={_testId}&connectionString={conn}");
    await new HttpClient().SendAsync(new HttpRequestMessage
    {
        Method = HttpMethod.Post,
        RequestUri = uri
    });
}

_coreRunner is a static variable were we store our running .NET CLI process so we only need to startup the process once per test run. If we are in debug mode we want to use a predefined port so that we manually can start the .NET 6 test service and debug the code. If we are not in debug a random port will be assigned for the service using a TCP listener.

private int GetFreePort()
{
    var l = new TcpListener(IPAddress.Loopback, 0);
    l.Start();
    int port = ((IPEndPoint)l.LocalEndpoint).Port;
    l.Stop();

    return port;
}

Then I point out the path to the test service, nothing fancy, but a bigger problem is that the .NET CLI are located on different places on our dev machines and on the build agents and also that the environment path for the CLI is not present on the build agent. So had to write some hacky code to make it work in both cases.

Once that was done it was just a matter of starting the dotnet cli with the run parameter and also supplying the port. In our case we needed to consume standard output otherwise Masstransit wouldn’t consume messages, go figure!

After that we need to ping the service and wait until its ready to receive commands. I also added a lot of debug output because we had trouble getting this to execute correctly on the build agents. Always nice to have some output even in the future.

When the test service is up and running we can call the setup method to set it up for a test. We store a Test ID that we can use to get the right IoC container on the service end. And that’s it.

Finally we need a method to execute commands and its pretty straight forward.

protected async Task ExecuteCommand<TCommand>(TCommand cmd) where TCommand : ICommand
{
    if(!_testId.HasValue) throw new ApplicationException($"You need to call {nameof(StartNetCoreTestRunner)} before calling {nameof(ExecuteCommand)}");

    var uri = new Uri($@"{GetCoreTestRunnerUrl()}ExecuteCommand?testId={_testId}&cmdType={typeof(TCommand).AssemblyQualifiedName}");
    await new HttpClient().SendAsync(new HttpRequestMessage
    {
        Method = HttpMethod.Post,
        RequestUri = uri,
        Content = new StringContent(JsonConvert.SerializeObject(cmd))
    });
}

Our DTO contract project has multiple target framework so it compiles both for .NET Standard 2.0 and .NET 6 meaning we can include the project in our legacy software which is nice for contract safety.
Right now black box testing is enough for us, we execute commands in the new system and assert outcome polling database in the legacy tests. Unit testing for the new code is done from unit tests in .NET 6 solution.

Almost forgot to talk about teardown. When the test have completed it need to call the service so that it can remove the instance.

if (_coreRunner != null && _testId.HasValue)
{
    var uri = new Uri($@"{GetCoreTestRunnerUrl()}TeardownTest?testId={_testId}");
    await new HttpClient().SendAsync(new HttpRequestMessage
    {
        Method = HttpMethod.Post,
        RequestUri = uri
    });
}

Also when all tests have ran we need to teardown the entire .NET service process.

[AssemblyCleanup]
public static void GlobalCleanup()
{
    if (_coreRunner != null && !_coreRunner.HasExited)
        _coreRunner.Kill();
}