The SVDA architecture

• Introduction
• The hardware tier
• The software tier
• The proxy layer
• The main server layer
• The client layer

The overall system architecture looks as follows:

The overall system architecture

The hardware tier

The hardware tier is composed of two types of devices: video and archive archive. A video server acts as a stream player and recorder. An archive sever fulfills a role of a library that stores all the data accessible for clients. The system can work with only one video and archive server present. Although, many video and archive servers shown on the picture that can be served by the system makes the architecture fully scalable. When the architecture composed of one video and one archive server, what would happened when the number of system's users would grow and too many of them would want to play a video at the same time? It might be impossible to store all data required by them on one video server. The system, which might be composed of many video and archive servers, can undoubtedly deal with this situation by adding a new video server into the architecture. The presented system performs load balancing and distributes videos among video servers and this way preserves its usefulness. The system doesn't specify the kind of video servers and archive libraries that can be compatible with it. It can operate with any video server and archive library - the software tier solves the problem of diversity.

The current implementation uses as a video server the Sun MediaCenter server that has a hardware, operating system and file system dedicated for multimedia. It is equipped with 50 GB disc array, can operate with MPEG-1 and MPEG-2 streams. It has a network card dedicated to Fast Ethernet, ATM, as well as analogue interfaces. The role of the archive server performs automated tape library ATL with total capacity of 2,64 TB, throughput of 13,5 GB/h. It supplies 3 independent drives with transfer rate of 1.25 MB/s and uses 264 cartridges of 10 GB each.

The software tier

The software tier is much more complicated. At the highest level of abstraction SVDA can be considered as consisting of three layers:
• The proxy layer, composed of VFS proxies, covering video servers, and AS proxies, covering storage devices;
• The main server layer, responsible for serving user requests and managing multimedia data;
• The client layer.

The proxy layer

Each device, either a video server or an archive library can communicate with other devices/applications through the video server or archive library specific way. Such diversity makes the use of them very complicated, and sometimes even impossible. If the system skipped the proxy layer and communicated with a server directly it would be unachievable to use another servers that are able to speak in a different way without modification of the main module.

VFS and AS proxies have been introduced into the system as the solution that improves the system's flexibility. They provide standard set of functions used to manipulate on contents of the servers, which they cover: adding, removing, changing attributes, getting system state etc. The proxy layer hides variety of vendor's solutions by means of CORBA interfaces. For instance, when a new archive library is to be established into a system, the only thing that must be performed is covering it by a proxy that implements the AS_proxy interface, which looks as follows:

//
// ....
//
interface AS_proxy_client
{
  strings get_list_of_files() raises (AS_proxy::AS_proxy_exception);
  file_status get_file_status(in string file_name)
    raises (AS_proxy::AS_proxy_exception);
  void delete_file(in string file_name)
    raises (AS_proxy::AS_proxy_exception);
  void rename_file(in string file_name_old, in string file_name_new)
    raises (AS_proxy::AS_proxy_exception);
  void retrieve_file(in lock_manager lockman, in string file_name,
      in string destination_IP, in unsigned short destination_port)
    raises (AS_proxy::AS_proxy_exception);
  void retrieve_file_out(in lock_manager lockman, in string file_name,
      out string source_IP, out unsigned short source_port)
    raises (AS_proxy::AS_proxy_exception);
  void stage_file(in string file_name)
    raises (AS_proxy::AS_proxy_exception);
  void stage_file_wait(in string file_name, in unsigned short nsecs)
    raises (AS_proxy::AS_proxy_exception);
  void store_file(in lock_manager lockman, in string file_name,
      in string source_IP, in unsigned short source_port)
    raises (AS_proxy::AS_proxy_exception);
};


The CORBA server that implements this interface communicates with the device using a protocol that is understood by this device. However, for the rest of the components of the system, it is accessible as a CORBA object with the same interface. Such a solution makes the system independent on any current and future solutions provided by any vendor.

In the current implementation we use, as an archive library, the ATL tape library, which gives the access to its resources by means of FTP. If one didn't possess such a fantastic device, it would be really easy for them to substitute it with a local, large-capacity file system that accesses its resources by common read/write operations. Such a replacement would have no influence on a system activity. What is more, this change would be even unrecognizable by the system, which would identify a new archive server as a CORBA object that implements the same CORBA interface.

The main server layer

The main server is the only part of the system directly visible for a client. A client connects to the system using standard CORBA techniques, and all its requests are then issued on the main server. Whenever a client wants to log into the system, insert a new film, request it or manage data hierarchy, he simply sends proper instructions to the server and it is its responsibility to process them correctly and notify a client about the results. A client has no idea of the number of video and archive servers that works with the system. He is even not aware of where the physical copies of a film are placed. All a client knows is whether he can play the film immediately or he has to wait some amount of time. A client has also the possibility of reserving the film at particular time, some protection of his access rights, hierarchical and easy access to directory and film structure etc. All that is provided by this system. The system has also more sophisticated functions, useful for the system administrator or an experienced user, enabling them to act on attached proxies or physical files. This functionality is accessible only for them because it must be used very carefully as it might do severe damage to the system when used without caution. An ordinary user has restricted view of the system but in return he receives a flexible and intuitive tool enabling film manipulation. An example of a user session and consecutive data flow is presented in this picture:

An example of a user session with SVDA
1. User requests a film from a client application at a particular time.
2. The main server checks whether a film is stored on a video server. If it isn't the servers downloads a file from an archive server (through AS proxy — 2a) and then stores it on a video server (2b).
3. Data transfer might be carried out through completely different network channel than this, which carries control messages.
4. When the time is up user orders the server to play a film. The system is almost sure that the film is present on the video server by means of previous (2) automatic downloading. The word `almost' was used because sometimes it is impossible to store all the required films due to too many requests performed by the system users. In that case the next step fails and the user is delivered with an appropriate message.
5. The main server issues the request on a video server that stores specified film.
6. The stream, that contains requested film, is transmitted to the client. If any video server just holds a film requested by a user, the steps 2 and 3 are omitted.

Film Management

Every film stored in the system is an abstract identity with no physical equivalent. And so is seen by a client as an abstract which is ready (or not) for playing. A client knows only its name and place in a hierarchical directory structure, owner and access rights. There are files stored on archive and video servers (called later instances) that are logically attached to films - so every film has one or more copies of itself currently present in the system. But these files are hidden from clients. This is because manual management of these instances can destroy the system consistency and should be restricted to minimum. The assumption in this system is that there is always a film instance on some of archive servers. If it is missing on a video server it will be downloaded - as video servers are used as a kind of cache. But if it is missing from an archive server, there is a danger that this film will be lost at all - as it can be removed from cache at any moment.

The main task of the whole system is to store new films and to provide existing ones to the client. Reacting for client's request is the core of the whole system. As mentioned previously, it was assumed, that disk space of video servers is not sufficient to store all data - only the tertiary storage device is large enough to keep them all. But a client cannot play it from an archive server directly and he cannot keep all he wants on video servers. So he has to make a reservation - he provides the time when he wants the film to be ready to play (i.e. copied on a video server) and hopes that the system will do everything possible to actually copy it there. And that is the point - the system must be able to distribute the resources in such a way that most needed films will be ready to play.

This task is performed by means of `priorities'. Each film has a priority associated with it. When a client requests a film at a specified time, the film becomes more important - its priority grows up. The priority is increasing when the time goes by. The maximum is reached, when the time for playing is up. The system stores on video servers only films with the highest priorities, those which are `most wanted'. If it is no space for the film that has been request on a video server, the file (files) with lower priorities (less important) are removed from the video server's file system and a new file is stored there. As the priority of the file grows when the time of playing is up, the probability of finding files with lower priorities, that might be deleted, grows. Such a mechanism makes the management of the files fully automatic.
Sometimes each file on every server is important (has a high priority). It happens when too many films have been requested and they cannot be stored in a limited video server's file system storage space. In such a situation there is no way to fulfill a client's request. When it happens very often, attaching a new video server to the system should be considered.

The main server's internal architecture

The internal structure of the main has been divided into several parts. The modular architecture that has been introduced into the system makes its implementation more stable and reliable. The architecture looks as follows:

The architecture of the main server

Each of the modules shown on the picture is responsible for various, separate system aspects:

User manager

Provides the most basic functionality connected with initial user login and authenticating him to the system.

Directory manager

Responsible for making available the hierarchical structure present in the system. The structure of directories, subdirectories and films is essential to make this system clear even for an inexperienced user and simplifying for example searching tasks.

System manager

Responsible for managing of AS and VFS proxies attached to the system. That means it manages the video and archive servers, determines which ones are active, which are not and enables the system administrator to attach new ones, detach those no longer necessary etc.

File manager

Provides a set of functions which enable manipulating on physical film instances stored on particular archive or video servers. As it was mentioned above, manual management of these files is dangerous for the system consistency and therefore is restricted only for root user. However it can sometimes be necessary to manually transfer some files from one server to another - when for example one server is to be detached from the system and the data stored there should be moved to another one.

Movie manager

Probably the most important of system parts as it provides the functionality which is essential from the user's point of view. This manager is responsible for performing various commands on key system components: the films. It is possible to insert new films, remove old ones, change their attributes etc. So this manager composes the majority of system elements which are made available to an ordinary end user.

Request manager

The salt of the whole Scalable Video Distributed Architecture: gives clients the possibility to request the films. A client can for example tell to the system: `I want to have film xxx ready for playing by tomorrow at 8.00am'. And this part of the system is responsible for doing its best to fulfill this request.

All of the presented parts have been covered by a CORBA interface. In this way, the system becomes accessible for all environments, which are able to communicate using CORBA middleware. The system is able to communicate with clients written in Java, C, C++, Perl and many other programming languages, and presents all the advantages that CORBA introduces into distributed environments.

The client layer

The CORBA interface that specifies the strict functionality of the server makes it easier to implement any client. It is worth repeating that a standard CORBA interface doesn't force any requirements upon any client - they can be developed using almost every programming language and so run on almost every platform.

As even the most perfect and useful server without any client is unusable for an end-user we have decided to develop a Java client in the current implementation of the system. The description how we can use it is described in The User guide.

Client architecture

These are the most important functions of the SVDA client:
• providing a user with the graphical representation of the video filesystem
• managing the filesystem via connection to the SVDA server
• streaming and playing the MPEG-1 system streams
The client's role in the system is shown in the figure below:



The SVDA client is made up of two parts: the filesystem manager and the player.

Filesystem Manager

The filesystem manager is responsible for the following jobs:
• connection to the SVDA server
• graphical user interface providing visual representation of the filesystem in a tree-like manner
• forwarding user requests to the server
• error handling
The ORB module in the figure above implements all the methods related to communication with servers. The communication is based on the CORBA IIOP protocol. It makes connections to two servers: the SVDA server and the MSM server. The SVDA server is used to manage the filesystem contents and the MSM server is required for streaming MPEG-1 files over the network. The module calls the servers' remote methods on behalf of the other client module shown in the picture - the SVDA module.

The SVDA module implements methods required for interaction with the user. It is responsible for the visual part of the client and for providing the user with the easy to operate GUI. It forwards all user requests regarding filesystem contents to the ORB module which then communicates with the appropriate server.

Player

JMF player is the second part of the client. It is a separate application used merely as an MPEG-1 video player. It is executed by the SVDA module in response for the "Play" operation request. The player is based on the JMF libraries and allows playing of MPEG-1 files as well as MPEG-1 streams encapsulated into UDP packets. It is capable of receiving unicast and multicast transmission of video streams. Since it is a separate application it can be used as a standalone player.

 

Table of Contents