rudironsoni dotnet-file-io
File I/O patterns for .NET applications. Covers FileStream construction with async flags, RandomAccess API for thread-safe offset-based I/O, File convenience methods, FileSystemWatcher event handling and debouncing, MemoryMappedFile for large files and IPC, path handling security (Combine vs Join), secure temp file creation, cross-platform considerations, IOException hierarchy, and buffer sizing guidance.
Scope
- FileStream construction with async flags
- RandomAccess API for thread-safe offset-based I/O
- FileSystemWatcher event handling and debouncing
- MemoryMappedFile for large files and IPC
- Path handling security (Combine vs Join) and secure temp files
Out of scope
- PipeReader/PipeWriter and network I/O -- see [skill:dotnet-io-pipelines]
- Async/await fundamentals -- see [skill:dotnet-csharp-async-patterns]
- Span/Memory/ArrayPool deep patterns -- see [skill:dotnet-performance-patterns]
- JSON and Protobuf serialization -- see [skill:dotnet-serialization]
- GC implications of memory-mapped backing arrays -- see [skill:dotnet-gc-memory]
Cross-references: [skill:dotnet-io-pipelines] for PipeReader/PipeWriter network I/O, [skill:dotnet-gc-memory] for POH and memory-mapped backing array GC implications, [skill:dotnet-performance-patterns] for Span/Memory basics and ArrayPool usage, [skill:dotnet-csharp-async-patterns] for async/await patterns used with file streams.
FileStream
Async Flag Requirement
FileStream async methods (ReadAsync, WriteAsync) silently block the calling thread unless the stream is opened with
the async flag. This is the most common file I/O mistake in .NET code.
// CORRECT: async-capable FileStream
await using var fs = new FileStream(
path,
FileMode.Open,
FileAccess.Read,
FileShare.Read,
bufferSize: 4096,
useAsync: true); // Required for true async I/O
byte[] buffer = new byte[4096];
int bytesRead = await fs.ReadAsync(buffer, cancellationToken);
```text
```csharp
// ALSO CORRECT: FileOptions overload
await using var fs = new FileStream(
path,
FileMode.Create,
FileAccess.Write,
FileShare.None,
bufferSize: 4096,
FileOptions.Asynchronous | FileOptions.SequentialScan);
```text
Without `useAsync: true` or `FileOptions.Asynchronous`, the runtime emulates async by dispatching synchronous I/O to the
thread pool -- wasting a thread and adding overhead.
### FileStreamOptions (.NET 6+)
```csharp
await using var fs = new FileStream(path, new FileStreamOptions
{
Mode = FileMode.Open,
Access = FileAccess.Read,
Share = FileShare.Read,
Options = FileOptions.Asynchronous | FileOptions.SequentialScan,
BufferSize = 4096,
PreallocationSize = 1_048_576 // Hint for write: reduces fragmentation
});
```text
`PreallocationSize` reserves disk space upfront when creating or overwriting files, reducing filesystem fragmentation on
writes.
---
## RandomAccess API (.NET 6+)
`RandomAccess` provides static, offset-based, thread-safe file I/O. Unlike FileStream, it has no internal position
state, so multiple threads can read/write different offsets concurrently without synchronization.
```csharp
using var handle = File.OpenHandle(
path,
FileMode.Open,
FileAccess.Read,
FileShare.Read,
FileOptions.Asynchronous);
// Thread-safe: offset is explicit, no shared position
byte[] buffer = new byte[4096];
int bytesRead = await RandomAccess.ReadAsync(
handle, buffer, fileOffset: 0, cancellationToken);
// Read from a different offset concurrently -- no locking needed
byte[] buffer2 = new byte[4096];
int bytesRead2 = await RandomAccess.ReadAsync(
handle, buffer2, fileOffset: 8192, cancellationToken);
```text
### Scatter/Gather I/O
```csharp
// Read into multiple buffers in a single syscall
IReadOnlyList<Memory<byte>> buffers = new[]
{
new byte[4096].AsMemory(),
new byte[4096].AsMemory()
};
long totalRead = await RandomAccess.ReadAsync(
handle, buffers, fileOffset: 0, cancellationToken);
```text
### When to Use RandomAccess vs FileStream
| Scenario | Use |
| ------------------------------------------------------- | ------------ |
| Concurrent reads from different offsets | RandomAccess |
| Sequential streaming reads/writes | FileStream |
| Index files, database pages, memory-mapped alternatives | RandomAccess |
| Integration with Stream-based APIs | FileStream |
---
## File Convenience Methods
For small files where streaming is unnecessary, the `File` static methods are simpler and correct.
```csharp
// Read entire file as string (small files only)
string content = await File.ReadAllTextAsync(path, cancellationToken);
// Read all lines
string[] lines = await File.ReadAllLinesAsync(path, cancellationToken);
// Stream lines without loading entire file (.NET 8+)
await foreach (string line in File.ReadLinesAsync(path, cancellationToken))
{
ProcessLine(line);
}
// Write text atomically (write-then-rename pattern not built-in)
await File.WriteAllTextAsync(path, content, cancellationToken);
// Read all bytes
byte[] data = await File.ReadAllBytesAsync(path, cancellationToken);
```text
### When Convenience Methods Are Appropriate
| File size | Approach |
| --------- | ---------------------------------------------------------- |
| < 1 MB | `File.ReadAllTextAsync` / `File.ReadAllBytesAsync` |
| 1--100 MB | `File.ReadLinesAsync` or FileStream with buffered reading |
| > 100 MB | FileStream or RandomAccess with explicit buffer management |
---
## FileSystemWatcher
### Basic Setup
```csharp
using var watcher = new FileSystemWatcher(directoryPath)
{
Filter = "*.json",
NotifyFilter = NotifyFilters.FileName
| NotifyFilters.LastWrite
| NotifyFilters.Size,
IncludeSubdirectories = true,
EnableRaisingEvents = true
};
watcher.Changed += OnChanged;
watcher.Created += OnCreated;
watcher.Deleted += OnDeleted;
watcher.Renamed += OnRenamed;
watcher.Error += OnError;
```text
### Debouncing Duplicate Events
FileSystemWatcher fires duplicate events for a single logical change (editors write temp file, rename, delete old).
Debounce with a timer.
```csharp
public sealed class DebouncedFileWatcher : IDisposable
{
private readonly FileSystemWatcher _watcher;
private readonly Channel<string> _channel;
private readonly CancellationTokenSource _cts = new();
public DebouncedFileWatcher(string path, string filter)
{
_channel = Channel.CreateBounded<string>(
new BoundedChannelOptions(100)
{
FullMode = BoundedChannelFullMode.DropOldest
});
_watcher = new FileSystemWatcher(path, filter)
{
EnableRaisingEvents = true
};
_watcher.Changed += (_, e) =>
_channel.Writer.TryWrite(e.FullPath);
}
// requires: using System.Runtime.CompilerServices;
public async IAsyncEnumerable<string> WatchAsync(
TimeSpan debounce,
[EnumeratorCancellation] CancellationToken ct = default)
{
using var linked = CancellationTokenSource
.CreateLinkedTokenSource(ct, _cts.Token);
var seen = new Dictionary<string, DateTime>();
await foreach (var path in
_channel.Reader.ReadAllAsync(linked.Token))
{
var now = DateTime.UtcNow;
if (seen.TryGetValue(path, out var last)
&& now - last < debounce)
continue;
seen[path] = now;
yield return path;
}
}
public void Dispose()
{
_cts.Cancel();
_watcher.Dispose();
_cts.Dispose();
}
}
```text
### Buffer Overflow
The internal buffer defaults to 8 KB. When many changes occur rapidly, the buffer overflows and events are lost.
Increase with `InternalBufferSize` (max 64 KB on Windows) and handle the `Error` event.
```csharp
watcher.InternalBufferSize = 65_536; // 64 KB
watcher.Error += (_, e) =>
{
if (e.GetException() is InternalBufferOverflowException)
{
logger.LogWarning("FileSystemWatcher buffer overflow -- events lost");
// Trigger full directory rescan
}
};
```text
### Platform Differences
| Platform | Backend | Notable behavior |
| -------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| Windows | ReadDirectoryChangesW | Most reliable; supports `InternalBufferSize` up to 64 KB |
| Linux | inotify | Watch limit per user (`/proc/sys/fs/inotify/max_user_watches`); recursive watches add one inotify watch per subdirectory |
| macOS | FSEvents (kqueue fallback) | Coarser event granularity; may batch events with slight delay |
---
## MemoryMappedFile
### Persisted (Large File Access)
Map a file into virtual memory for random access without explicit read/write calls. Efficient for large files that do
not fit in memory -- the OS pages data in and out as needed.
```csharp
using var mmf = MemoryMappedFile.CreateFromFile(
path,
FileMode.Open,
mapName: null,
capacity: 0, // Use file's actual size
MemoryMappedFileAccess.Read);
using var accessor = mmf.CreateViewAccessor(
offset: 0,
size: 0, // Map entire file
MemoryMappedFileAccess.Read);
// Read a struct at a specific offset
accessor.Read<MyHeader>(position: 0, out var header);
// Or use a view stream for sequential access
using var stream = mmf.CreateViewStream(
offset: 0,
size: 4096,
MemoryMappedFileAccess.Read);
```text
### Non-Persisted (IPC Shared Memory)
```csharp
// Process A: create shared memory region
using var mmf = MemoryMappedFile.CreateNew(
"SharedRegion",
capacity: 1_048_576, // 1 MB
MemoryMappedFileAccess.ReadWrite);
using var accessor = mmf.CreateViewAccessor();
accessor.Write(0, 42);
// Process B: open existing shared memory region
using var mmf2 = MemoryMappedFile.OpenExisting(
"SharedRegion",
MemoryMappedFileRights.Read);
using var accessor2 = mmf2.CreateViewAccessor(
0, 0, MemoryMappedFileAccess.Read);
int value = accessor2.ReadInt32(0); // 42
```text
For GC implications of memory-mapped backing arrays and POH usage, see [skill:dotnet-gc-memory].
---
## Path Handling
### Path.Combine vs Path.Join Security
`Path.Combine` silently discards the first argument when the second argument is a rooted path. This enables path
traversal attacks when user input is passed as the second argument.
```csharp
// DANGEROUS: Path.Combine drops basePath when userInput is rooted
string basePath = "/app/uploads";
string userInput = "/etc/passwd";
string result = Path.Combine(basePath, userInput);
// result = "/etc/passwd" -- basePath is silently ignored
// SAFER: Path.Join does not discard on rooted paths (.NET Core 2.1+)
string result2 = Path.Join(basePath, userInput);
// result2 = "/app/uploads//etc/passwd" -- preserves basePath
```text
### Path Traversal Prevention
```csharp
public static string SafeResolvePath(string basePath, string userPath)
{
## Detailed Examples
See [references/detailed-examples.md](references/detailed-examples.md) for complete code samples and advanced patterns.