File Splitter and Merger

In a recent job interview I was given a whiteboard coding challenge in which I had to write some pseudo-code in C# for splitting a file. I hadn’t worked with System.IO for a long time so I thought I’d create a console tool for doing this with .NET 5. This is how FileSplitterMerger was born a few days later.

This went from being a simple file splitter with a fixed number of chunks, to handling splitting with number of chunks as a parameter, then splitting with chunks of a specified size as a parameter, and then merging. I had fun doing this. The following are the main technical decisions I made for this project:

• Config: Dependency Injection of IConfiguration
• Handling large files (20GB+): Using streams and buffers to read and write to disk sequentially
• Optimising performance for large files: Using Process.PrivateMemorySize64 to dynamically adapt the buffer size to the available machine memory for the process.
• Dealing with arguments: Implemented a custom argument parser which uses System.ComponentModel.DescriptionAttribute to enrich enums so they can be decorated with a key and a value (where the key is the argument flag and the value is the argument value.

The beauty of .NET 5 is it’s multi-platform so I tested this both on Windows and on WSL. It’s great to see this handle huge files. I used “fsutil file createnew” to create some 30GB+ dummy files for testing purposes.