The purpose of this project is to design and implement a distributed storage system. The system's architecture is inspired by the abstract architecture for distributed file systems. Among the components, we have a client and multiple instances of a server. We adopt an asynchronous peer-to-peer replication among the server instances with a replication factor of three. The system supports two operations: read and write objects. Each object is represented in a JSON format and stored in a local file for each instance of the server. Note that the files have a limited size of 30 MB. As well each object has a key (a string) attached to it. Finally, we use Kafka as a middleware between the client and the server(s) To write an object, the JSON content and the key are sent to the server through Kafka. When the write operation succeeds, the client receives an
Question:
The purpose of this project is to design and implement a distributed storage system. The system's architecture is inspired by
the abstract architecture for distributed file systems.
Among the components, we have a client and multiple instances of a server. We adopt an asynchronous peer-to-peer
replication among the server instances with a replication factor of three. The system supports two operations: read and write
objects. Each object is represented in a JSON format and stored in a local file for each instance of the server. Note that the files
have a limited size of 30 MB. As well each object has a key (a string) attached to it. Finally, we use Kafka as a middleware
between the client and the server(s)
To write an object, the JSON content and the key are sent to the server through Kafka. When the write operation succeeds, the
client receives an acknowledgement from each server. A write operation with an existing key replaces the existing object.
Moreover, the client sends a request through Kafka and receives a copy from each server with the logical clock attached to it to
read an object. The client then analyses the logical clocks, decides upon the latest version and prints it out. In case there are
multiple concurrent versions, it prints them all out. When the client identifies that a given copy has a version older than the other
versions, it notifies the replica, which then updates its store. Finally, the replicas keep a direct connection to each other and
exchange a heartbeat message regularly with their logical clock piggybacked to the message. In this implementation, we will
adopt
The task is the implementation of the system, including the setup of the Kafka cluster.
Setup of the Kafka cluster, including topic management.
Implementation of the client and services in the Ballerina language.
Step by step
Solved in 2 steps with 1 images
