15. Streams and I/O || C# 10

Question 1

Can you name 3 concepts on which the .NET stream architecture centers?

Answer 1

1. Backing stores
2. decorators
3. adapters

Question 2

What is a ‘backing store’?

Answer 2

A backing store is the endpoint that makes input and output useful, such as a file or network connection. Precisely, it is either or both of the following:

• A source from which bytes can be sequentially read;
• A destination to which bytes can be sequentially written;

Question 3

What is a Stream?

Answer 3

A stream exposes a standard set of methods for reading, writing and positioning. Unlike an array, for which all the backing data exists in memory at once, a stream deals with data serially - either one byte at a time or in blocks of manageable size. Hence, a stream can use a small, fixed amount of memory regardless of the size of its backing store.
The abstract Stream class is the base for all streams. It defines methods and properties for three fundamental operations: reading, writing and seeking;

Question 4

How can we categorize streams?

Answer 4

Streams fall into two categories:

• Backing store streams: These are hardwired to a particular type of backing store, such as FileStream or NetworkStream;
• Decorator streams: These feed off another stream, transforming the data in some way, such as DeflateStream or CryptoStream;

To summarize, backing store streams provide the raw data, decorator streams provide transparent binary transformations such as encryption

Question 5

Can you name the benefits of Decorator streams?

Answer 5

Decorator streams have the following architectural benefits:
1. They liberate backing store streams from needing to implement such features as compression and encryption themselves;
2. Streams don’t suffer a change of interface when decorated;
3. You connect decorators at runtime;
4. You can chain decorators together (e.g. a compressor followed by an encryptor);

Question 6

What is an adaptor in streams as a topic?

Answer 6

If we consider that backing stores and decorator streams deal exclusively in bytes, when applications often work at higher level, such as text or XML, we need some sort of translation between the types.
Adaptors bridge this gap by wrapping a stream in a class with specialized methods typed to a particular format. For example, a text reader exposes ReadLine method, an XML writer exposes a WriteAttributes method.
An adaptor wraps a stream, just like a decorator. Unlike a decorator, however, an adapter is not itself a stream; it typically hides the byte-oriented methods completely.
Adapters offer typed methods for dealing with higher level types such as strings and XML.

Question 7

Explain how the Read method in Streams work

Answer 7

Read receives a block of data from the stream into an array. It returns the number of bytes received, which is always either less than or equal to the count argument. If it’s less than count, it means that either the end of the stream has been reached or the stream is giving you the data in smaller chunks (as is often the case with network streams). In either case, the balance of bytes in the array will remain unwritten, their previous values preserved.
With Read you can be certain you’ve reached the end of the stream only when the method returns 0

Question 8

What does the Seek method allows to do?

Answer 8

The Seek method allows you to move relative to the current position or the end of the stream.
With nonseekable stream (such as an encryption stream) the only way to determine its length is to read it completely through. Furthermore if you need to reread a previous section, you must close the stream and start fresh with a new one.

Question 9

Describe disposal semantics of the stream

Answer 9

In general streams follow sandard disposal semantics:

• Dispose and Close are identical in function;
• Disposing or closing a stream repeatedly causes no error.

Closing a decorator stream closes both the decorator and its backing store stream. With a chain of decorators, closing the outermost decorator closes the whole lot.

Question 10

Are streams thread-safe?

Answer 10

As a rule, streams are not thread-safe, meaning two threads cannot concurrently read or write to the same stream without possible error.
As a workaround there is a Synchronized method, that returns a thread-safe wrapper. The wrapper works by obtaining an exclusive lock around each read, write or seek, ensuring that only one thread can perform such an operation at a time.

Question 11

FileStream class, besides others, has two methods - OpenWrite and Create. Both of them write content into the file, but what are the differences?

Answer 11

The differences shows up when we try to write something into a file that already have some content written in it. Create method truncates any existing content while OpenWrite leaves existing content intact with the stream positioned at zero. If you write fewer bytes than were previously in the file, OpenWrite leaves you with a mixture of old and new content.

Question 12

What’s the difference between ReadLines and ReadAllLines?

Answer 12

They do the same thing except ReadLines returns lazily evaluated IEnumerable<string>. This is more efficient because it doesnt load the entire file into memory at once.

Question 13

What is a MemoryStream?

Answer 13

MemoryStream uses an array as a backing store. This partly defeats the purpose of having a stream because the entire backing store must reside in memory at once. MemoryStream is still usefull when you need random access to a nonseekable stream.
You can conver MemoryStream into byte array by calling ToArray. The GetBuffer method does the same job more efficiently by returning a direct reference to the underlying storage array.
Also fun fact that calling Flush on MemoryStream does absolutely nothing :)

Question 14

What is a PipeStream?

Answer 14

PipeStream provides a simple means by which one process can communicate with another through the operating system’s protocol. There are two kinds of pipe:
1. Anonymous pipe (faster):
Allows one-way communication between a parent and child process on the same computer;
2. Named pipe (more flexible):
Allows two-way communication between arbitrary processes on the same computer or different computers across a network.
A pipe is good for interprocess communication on a single computerL it doesn’t rely on a network transport, which means no network protocol overhead, and it has no issues with firewalls.

A pipe is good for interprocess comunication on a single computer: it doesn’t rely on a network transport, which means no network protocol overhead, and it has no issues with firewalls.

Pipes are stream based, so one process waits to receive a series of bytes while another process sends them.

Question 15

Define PipeStream class

Answer 15

PipeStream is an abstract class with four concrete subtypes. Two are used for anonymous pipes and other two for named pipes.
Named pipes are simpler to use.

Question 16

What are Named Pipes?

Answer 16

With the named pipes, the parties communicate through a pipe of the same name. The protocol defines two distinct roles: the client and server. Communication happens between the client and server as follows:
* The server instantiates a NamedPipeServerStream and then calls WaitForConnection;
* The client instantiates a NamedPipeClientStream and then calls Connect (with an optional timeout)

The two parties then read and write the streams to communicate.
~~~
(using var s = new NamedPipeServerStream(“pipedream”)){
s.WaitForConnection();
s.WriteByte(100); // Send the value 100
Console.WriteLine(s.ReadByte());
}

(using var c = new NamedPipeClientStream(“pipedream”)){
c.Connect();
Console.WriteLine(s.ReadByte());
c.WriteByte(200); // Send the value 200 back
}
~~~

Named pipes are bidirectional by default, so either party can read or write their stream. This means that the client and server must agree on some protocol to coordinate their action, so both parties don’t end up sending or receiving at once.

Question 17

What is a message transmission and why do we need it?

Answer 17

Since you cannot determine whether a PipeStream has finished reading a message simply by waiting for Read to turn 0, you need to agree between client and server on the length of the message. To help with messages longer than one byte, pipes provide a message transmission mode. If this is enabled a party calling Read can know when a message is complete by checking the IsMessageComplete property.

(using var s = new NamedPipeServerStream("pipedream", PipeDirection.InOut, 1, PipeTransmissionMode.Message)){
    s.WaitForConnection();
    byte[] msg = Encoding.UTF8.GetBytes("Hello");
    s.Write(msg, msg.Length);

    Console.WriteLine(Encoding.UTF8.GetString(ReadMessage(s)));
    }

(using var c = new NamedPipeClientStream("pipedream")){
    c.Connect();
    c.ReasMode = PipeTransmissionMode.Message;
    Console.WriteLine(Encoding.UTF8.GetString(ReadMessage(c)));
    byte[] message = Encoding.UTF8.GetBytes("Hello right back!");
    c.Write(message, 0, message.Length);

Question 18

What are anonymous pipes and what are they used for?

Answer 18

An anonymous pipe provides a one-way communication stream between a parent and child process. Instead of using a system-wide name, anonymous pipes tune in through a private handle.
As with named pipes, there are distinct client and server roles. The system of communication is a little different, however, and proceeds as follows:
* The server instantiates an AnonymousPipeServerStream, commiting to a PipeDirection of In or Out.
* The server calls GetClientHandleAsString to obtain an identifier for the pipe, which it then passes to the client;
* The child process instantiates an AnonymousPipeClientStream, specifying the opposite PipeDirection;
* The server releases the local handle that was generated in Step 2, by calling DisposeLocalCopyOfClienthandle;
* The parent and child processes communicate by reading/writing the stream;
Because anonymous streams are unidirectional, a server must create two pipes for bidirectional communication.
As with named pipes, the client and server must coordinate their sending and receiving and agree on on the lenght of each transmission. Anonymous pipes don’t support message mode.

Here is an analogy from chatGPT :)
Imagine you have two people, Alice and Bob, who want to communicate, but they can only send messages in one direction. They each have their own dedicated messenger to send messages to the other person. In this modified analogy:

1. Alice represents one part of your C# program (e.g., the server).
2. Bob represents another part of your C# program (e.g., the client).
3. Two Messengers represent the two sets of anonymous pipes, one for each direction of communication (Alice’s messenger for sending messages to Bob, and Bob’s messenger for sending messages to Alice).

Here’s how it works:

1. Sending Messages from Alice to Bob (Alice’s Messenger): When Alice wants to send a message (data) to Bob, she writes the message on a note (puts data into her messenger’s pipe). Her dedicated messenger (pipe) takes the note and delivers it to Bob.
2. Receiving Messages from Alice (Bob’s Messenger): Bob, on the other hand, has his own messenger (pipe) to receive messages from Alice. His messenger (pipe) delivers the note (data) to Bob, and he reads the message (data) written on it.
3. Sending Messages from Bob to Alice (Bob’s Messenger): If Bob wants to send a message (data) to Alice, he writes it on a note and hands it to his dedicated messenger (pipe). The messenger takes the note and delivers it to Alice.
4. Receiving Messages from Bob (Alice’s Messenger): Alice, in turn, has her own messenger (pipe) to receive messages from Bob. Her messenger (pipe) delivers the note (data) to Alice, and she reads the message (data) written on it.

So, in the case of anonymous pipes in C#, you need two separate sets of pipes: one for sending data in each direction of communication. This unidirectional setup allows you to achieve bidirectional communication by using two sets of pipes—one for each direction—similar to how Alice and Bob each have their dedicated messengers for sending messages to the other person.

Question 19

What is a BufferedStream? What buffering gives to a stream?

Answer 19

BufferedStream decorates or wraps another stream with buffering capabilities and it is one of a number of decorator stream types in .NET.
Buffering improves performance by reducing round trips to the backing store.

Question 20

Why would you need a BufferStream together with FileStream if it already comes with built-in buffering?

Answer 20

In this case its only use might be in enlarging the buffer on an already-constructed FileStream.

Question 21

Why do we need Stream Adapters?

Answer 21

A Stream deals only in bytes; to read or write data types such as strings, integers or XML elements, you must plug in an adapter; Here are the main catogories:
* Text adapters (for string and character data): TextReader, TextWriter, StreamReader, StreamWriter, StringReader, StringWriter;
* Binary adapters (for primitive types such as int, bool, string and float): BinaryReader, BinaryWriter;
* XML adapters: XmlReader, XmlWriter;

Question 22

Can you name one additional difference between the TextReader/TextWriter and StreamReader/StreamWriter that is not just the type of data accepted?

Answer 22

TextReader and TextWriter are by themselves just abstract classes with no connection to a stream or backing store. The StreamReader and StreamWriter types, however, are connected to an underlying byte-oriented stream, so they must convert between characters and bytes. They do so through Encoding class, ehich you choose when constructing StreamReader or StreamWriter.

Question 23

What is the simpliest of encodings and why?

Answer 23

The simpliest of encodings is ASCII because each character is represented by one byte.

Question 24

Do you know why in UTF encoding first 127 encodes to a single byte, but remaining characters encode to a variable number of bytes?

Answer 24

Since the simpliest encondings is ASCII and it takes those first 127 characters of the Unicode set into its single byte, the UTF-8 also does the same for ASCII compatibility, but the rest characters then takes other amount of memory.

Question 25

When speaking about bytes, can you explain how it is stored in memory? What is a memory address?

Answer 25

In computer memory, data is stored in a sequential manner, and each byte of data has a unique address that identifies its location in memory. These addresses are used by the computer’s memory management system to access and manipulate data. When discussing the “lowest” and “highest” memory addresses, we’re referring to the relative positions of these addresses in memory.

1. Lowest Memory Address: The lowest memory address refers to the address where the computer starts storing data. It’s typically the address of the first byte in memory. In most computer architectures, memory addresses start at zero or some other low value and increment as you move through memory. So, the lowest memory address is the address of the first byte, and it has the smallest value.
2. Highest Memory Address: The highest memory address, on the other hand, refers to the address of the last byte in memory. It’s the address that marks the end of the available memory space. The value of the highest memory address is typically the maximum address that the computer’s memory management system can access.

When we talk about storing a single byte of data in memory, it occupies one memory address. The concept of “least significant byte” (LSB) and “most significant byte” (MSB) becomes more relevant when you’re dealing with multi-byte data types, such as integers or floating-point numbers, which span multiple memory addresses.

For example, in a little-endian system (like x86 architecture), if you were to store a 32-bit integer in memory, it would be stored as follows:

Lowest Memory Address:    Byte 0 (LSB)   Byte 1   Byte 2   Byte 3 (MSB)    Highest Memory Address
                         |  00000001   |  00100011 |  01000101 |  01100111  |

In this representation, each byte is stored at a sequential memory address, with the least significant byte (LSB) at the lowest memory address (Byte 0) and the most significant byte (MSB) at the highest memory address (Byte 3).

So, when we talk about bytes being stored at different memory addresses, it’s mainly relevant when dealing with multi-byte data types that span multiple memory locations, and the terms “least significant” and “most significant” refer to the order of those bytes within the data type. For single-byte data, there’s no distinction between least and most significant bytes.

Question 26

What does it mean when two-byte pairs are written in a “little-endian” or “big-endian” order?

Answer 26

It means that the order of the bits in a byte vary:
“little-endian” - least significant bit first (default)
“big-endian” - most significant bit first.

Question 27

How can you tear down stream adapters?

Answer 27

We have 4 ways of doing so:
1. Close the adapter only;
2. Close the adapter and then close the stream;
3. (For writers) Flush the adapter and then close the stream;
4. (For readers) Close just the stream;

Close and Dispose are synonymous with adapters, just as they arer with streams;

(1) and (2) are semantically identical because closing an adapter automatically closes the underlying stream. Good example is when we nest “using” statements we implicitly taking option (2):
~~~
using (FileStream fs = File.Create(“test.txt”))
using(TextWriter writer = new StreamWriter(fs))
writer.WriteLine(“Line”);
~~~

Question 28

What is the general rule with the Compression streams?

Answer 28

• You always write to the stream when compressing;
• You always read from the stram when decompressing;

Question 29

How many ways do you know to compress data?

Answer 29

In C#, you have three options for compressing data: DeflateStream, GZipStream, and ZipArchive/ZipFile. Each of these options serves a specific purpose and has its own set of features and use cases. Let’s explore the differences and when to use each one:

1. DeflateStream:
   
   • DeflateStream is a stream-based compression class that uses the DEFLATE compression algorithm.
   • It provides relatively fast compression and decompression and is suitable for compressing individual files or streams of data.
   • DEFLATE is a lossless compression algorithm that works well for a wide range of data types.
   • Use DeflateStream when you need basic compression for a single file or stream and want a simple and efficient solution.
2. GZipStream:
   
   • GZipStream is also a stream-based compression class, but it uses the GZIP compression format, which is built on top of DEFLATE.
   • GZIP includes additional metadata such as file names and timestamps, making it suitable for compressing collections of files or streams along with their associated information.
   • GZIP is commonly used for compressing files in formats like .gz and is often used in web applications to compress HTTP responses.
   • Use GZipStream when you need to compress multiple files or streams with associated metadata or when working with GZIP-compressed files.
3. ZipArchive and ZipFile:
   
   • ZipArchive and ZipFile are classes for creating and manipulating ZIP archives, which are containers for compressed files and directories.
   • ZIP archives can contain multiple compressed files and directories, and they provide a convenient way to package and compress data for storage or distribution.
   • These classes allow you to add, extract, and manage the contents of ZIP archives easily.
   • Use ZipArchive and ZipFile when you need to work with collections of files and directories that need to be compressed together into a single archive.

In summary:

• Use DeflateStream for simple compression of individual files or streams.
• Use GZipStream when you want to compress multiple files or streams along with associated metadata, or when working with GZIP-compressed data.
• Use ZipArchive and ZipFile when you need to create, manipulate, or extract files and directories within a ZIP archive.

The choice between these options depends on your specific use case and the requirements of your application. If you’re unsure which one to use, consider the nature of your data and whether you need to compress individual items or package multiple items together in an archive.

Question 30

Describe the difference of data that is kept in ApplicationData, LocalApplicationData and CommonApplicationData folders

Answer 30

ApplicationData: settings that travel with a user across a network;
LocalApplicationData: nonroaming data (specific to logged-in user);
CommonApplicationData: shared data by every user of the computer;

Question 31

What are the benefits of memory-mapped files?

Answer 31

Memory-mapped files provide two key features:
1. Efficient random access to file data;
2. The ability to share memory between different processes on the same computer;

These files work by wrapping the operating system’s API for memory-mapped files.

Question 32

Which is faster in file I/O - memory-mapped files or FileStreams?

Answer 32

Although an ordinary FileStream allows random I/O (by setting the stream’s Position property), it’s optimized for sequential I/O. As a rough rule of thumb:
* FileStream are approximately 10 times faster than memory-mapped files for sequential I/O;
* Memory-mapped files are approximately 10 times faster than FileStream for random I/O;