IAsyncEnumerable is C#'s answer to streaming data asynchronously, and the most surprising thing about it is that it doesn’t actually stream data itself; it’s a contract that says "I can give you items, one by one, when you ask for them, without blocking."
Let’s see it in action. Imagine we’re fetching a list of user IDs from a slow, remote API, and for each ID, we’re going to perform some potentially long-running operation, like fetching detailed user data. We don’t want to wait for all IDs to be fetched before we start processing, nor do we want to block the thread while fetching each user’s details.
using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Threading.Tasks;
public class UserService
{
private readonly HttpClient _httpClient;
public UserService(HttpClient httpClient)
{
_httpClient = httpClient;
}
public async IAsyncEnumerable<string> GetUserNamesAsync(IEnumerable<int> userIds)
{
foreach (var userId in userIds)
{
// Simulate fetching user data from a remote API
await Task.Delay(100); // Simulate network latency
var userName = await FetchUserNameByIdAsync(userId);
yield return userName; // Yield the result asynchronously
}
}
private async Task<string> FetchUserNameByIdAsync(int userId)
{
// In a real scenario, this would be an HTTP call to a backend service.
// For demonstration, we'll just return a dummy name.
await Task.Delay(50); // Simulate more network latency
return $"User_{userId}";
}
}
public class Program
{
public static async Task Main(string[] args)
{
var httpClient = new HttpClient();
var userService = new UserService(httpClient);
var userIds = new List<int> { 1, 2, 3, 4, 5 };
Console.WriteLine("Starting to fetch user names...");
await foreach (var userName in userService.GetUserNamesAsync(userIds))
{
Console.WriteLine($"Received: {userName}");
// Simulate processing the user name
await Task.Delay(200);
}
Console.WriteLine("Finished fetching all user names.");
}
}
In this example, GetUserNamesAsync is an async method that returns IAsyncEnumerable<string>. The yield return statement inside the loop is what makes this an asynchronous iterator. When the await foreach loop in Main requests the next item, the GetUserNamesAsync method resumes execution from where it left off, performs the await Task.Delay(100), calls FetchUserNameByIdAsync, and then yields the result. The Main method then processes this userName (simulated by another Task.Delay(200)), and only then does it request the next item from GetUserNamesAsync. This happens without blocking any threads.
The problem this solves is that of processing large or infinite sequences of data where fetching each item might be an I/O-bound operation. Before IAsyncEnumerable, you might have had to:
- Fetch all data into memory: This is bad for large datasets.
- Use callbacks or event-based patterns: This can lead to complex, unreadable code, especially when dealing with multiple asynchronous operations.
- Use blocking I/O: This defeats the purpose of asynchronous programming and can severely impact scalability.
IAsyncEnumerable provides a familiar foreach-like syntax for asynchronous iteration, making the code much cleaner and easier to reason about. Internally, IAsyncEnumerable<T> is an interface with a single method, GetAsyncEnumerator(), which returns an IAsyncEnumerator<T>. This enumerator has MoveNextAsync() and Current properties, similar to IEnumerator<T>, but MoveNextAsync() returns a Task<bool> indicating whether there’s a next element. The await foreach syntax is syntactic sugar that abstracts away the manual calls to MoveNextAsync() and checking its result.
The real power comes when you chain multiple asynchronous operations. Consider a scenario where you’re fetching a list of product IDs, then for each ID, fetching product details, and then for each product detail, fetching its inventory.
public class ProductService
{
public async IAsyncEnumerable<int> GetProductIdsAsync()
{
await Task.Delay(100); // Simulate fetching IDs
yield return 101;
await Task.Delay(100);
yield return 102;
}
public async IAsyncEnumerable<ProductDetails> GetProductDetailsAsync(int productId)
{
await Task.Delay(200); // Simulate fetching details
yield return new ProductDetails { Id = productId, Name = $"Product {productId}" };
}
public async Task<int> GetInventoryCountAsync(int productId)
{
await Task.Delay(50); // Simulate fetching inventory
return productId % 2 == 0 ? 10 : 0;
}
}
public class ProductDetails { public int Id; public string Name; }
// In Main method:
var productService = new ProductService();
await foreach (var productId in productService.GetProductIdsAsync())
{
await foreach (var productDetail in productService.GetProductDetailsAsync(productId))
{
var inventory = await productService.GetInventoryCountAsync(productDetail.Id);
Console.WriteLine($"Product: {productDetail.Name}, Inventory: {inventory}");
}
}
This nested await foreach structure elegantly handles multiple levels of asynchronous data fetching.
A common point of confusion is how yield return works in async iterators. Unlike regular iterators where yield return immediately returns a value, in an async iterator, yield return enqueues the value to be returned by the next call to MoveNextAsync(). The await operations before the yield return are what pause the execution of the iterator’s method body. When MoveNextAsync() is called again, the method resumes after the await and continues until the next yield return or the end of the method. This means the control flow is managed by the enumerator, allowing the caller to process items as they become available without the producer being blocked.
The next concept you’ll likely encounter is how to handle cancellation with IAsyncEnumerable using CancellationToken.