Project Goal

To enable the of use standard binary packages and package managers to deploy the Kaltura platform.

• This project's sole purpose is to create standard installation packages for Kaltura CE. It will make the installation process far easier (compared to the existing PHP/bash installers) and allow users to use package managers (aptitude, yum, synaptic, kpackage, etc.) and install Kaltura as you would any other binary package in your operating system of choice.
• A longer term goal of this project is to allow standard distribution of the Kaltura Video Platform via the official distribution repositories.

Latest Project Status

Why do we need package installers?

deb and RPM are the de-facto standard for binary installations on Linux. Using this installation method will yield the following advantages to Kaltura admins:

• Compatibility with the industry standards for installing Linux packages, simplifying prerequisites and platform installation and wider and easier package distribution.
• Simplified upgrade flow using standard package manager commands.
• Simplified patch deployment using standard package manager commands.
• Clean and organized components versioning and changes-log using standard package manager commands.
• Faster configuration post system upgrade - When upgrading, package managers would show a diff between current configuration files and the ones in the package.

Is this project only Linux focused?

Since Kaltura is built mostly in PHP and its major components are available for multiple OSes, it is theoretically possible to deploy Kaltura on any OS.
In the longer term we would love for this project to expand beyond deb and RPM packages to other distributions and operating systems.
However, since the majority of Kaltura users and, majority of servers on the web, are Linux based, this project is started off focusing only on deb and RPM packages.
We will try as much as possible to keep a common folder and coding practices that will allow for easier porting later.
If you are using Kaltura on another distribution or operating system, and have experience in building installation packages - your help and contribution to the project will be greatly appreciated!
if you'd like to join and contribute other package types.

Which Linux distributions will be supported?

Following results from a recent Kaltura Community usage survey, we've set this project to target the most popular Linux distros among Kaltura users, which includes:

• RHEL/CentOS 6 and above Install Kaltura on RH based systems.
• Ubuntu 12.04 and above.
Install Kaltura on deb based systems.
• Debian Wheezy and above.
Install Kaltura on deb based systems.

Project Phases

Phase A - Specifications & Definitions

Creating the package specifications, we intend to break it down to different packages to achieve a modular setup allowing the ability to install only specific needed platform components, and to build a structure that will enable a cluster environment, distributed servers platform deployment and easier component updates/patching.

The Kaltura Packages Are

• Kaltura Core (aka kaltura-base) - files and actions needed by all Kaltura server roles.
• Kaltura API node (aka kaltura-front) - files and actions needed for an API-Front server role.
• Kaltura Batch Server (aka kaltura-batch) - files and actions needed for asynchronous batch processing (transcoding, metadata extraction, emails, notifications, etc.) Kaltura server role.
• Kaltura Sphinx node (aka kaltura-sphinx) - files and actions needed for a Sphinx Kaltura Indexing server role.
• Kaltura DB node - files and actions needed for a MySQL DB Kaltura server role.
• Kaltura Data Warehouse (aka kaltura-dwh) - files and actions needed for the Kaltura analytics data warehouse server role.
• Kaltura Streaming Server - This package configures the files required to run the base Kaltura Media Server handling all actions for creation and management of live streams using Kaltura.
• Kaltura VOD packager - This package configures the NGINX-based VOD Packager NGINX-based VOD Packager.
• Red5 Streaming Server - files and actions needed for Red5 server configuration to be used by the Kaltura Platform.
• Meta-Package Kaltura Server (aka kaltura-server) - This will be a meta package linking all other required packages to run an All-In-One Kaltura environment.

Phase B - Building The Packages

At this phase the packages will be built according to each distro guidelines and requirements.
Debian, Ubuntu, Fedora.

You can track the development, contribute and help review the packages on the GitHub repository.

Phase C.1 - Writing A Post-Install Configuration Guides

While the packages will enable a complete installation of the system, post-installation configuration is always required, especially in clustered/multi-server environments. At this phase we will perform initial testing, and focus on preparing all necessary instructions into a post-install system configuration guide for the sys-admin wanting to deploy Kaltura.

For RPM, we have the following guides ready:

• Installing Kaltura on RedHat Linux.
• Setting up Kaltura platform monitoring.
• Deploying Kaltura Clusters.

Phase C.2 - Alpha Repositories & Testing

Announcing public repository URL and calling for testers
At this phase admins who are not necessarily developers, and beta testers are welcome to run the package installation using package managers of their liking, submit bugs and feature requests, and help make the packages stable and ready for public release.

Also in this phase - creation of Chef deployment automation scripts and preparing the definitions for the CI (Continuous Integration) system.

• Deploying Kaltura using Opscode Chef.

Phase D - Public Beta, and Continuous Integration System

Track the development and learn more, visit the Continuous Integration System Project.

The creation and deployment of a CI (Continuous Integration) system for building, testing and distributing the platform packages.
This is a crucial phase in the maturity of the project, which will allow us to determine health status of the packages before releasing to user QA and public distribution.

During this phase beta testing of various cluster deployments will be continued, and tools to deploy and verify deployment health will be created.

Phase E - RC and GA Releases

Updating Kaltura.org website, deletion of the old installer repositories, public announcements and overall clean-up.

Calling the wider Kaltura admins in the community to install and migrate to the new packages.

Phase F - Post-Install Script for Developer Environment

Create a post install script that will pull updated source-code from the official github repositories and configure an all-in-one developer environment for core developers wanting to write server plugins or modify and test core files.

Phase G - Distribution to Official Distro Repositories

Submit official requests to each official distro repositories (Ubuntu, Fedora and Debian).

How can you help and contribute?

This project has an open call-for-participation even before a single line of code was written. Why not wait till there’s at least an early alpha? Because the installation of Kaltura has been a major hurdle for everyone in the community, and we want to make sure you are involved from the very beginning - the deeper you get with the code, the better and more stable this project will be!
On a weekly basis the community team will update this page with project progress, so you can stay up to date, but you can also watch the github repositories and get updates when new code is pushed.

There will be a weekly IRC (#kaltura on freenode.net) meeting every Monday at 10am NY time, where we openly discuss project status and details.
Join us at #kaltura on freenode and share your feedback, insights and ideas for the project!

If you've already dipped your toes in the code, have experience building Linux packages and want to join the team - .

To submit bugs, patches or feature requests, please use the issue tracker,
and make sure to tag your issue with the proper label to the package type you're testing/extending:

• RPM Package Issues Tracker.
• deb Package Issues Tracker.

Project Questions & Answers

• • Under Specifications & Definitions, why did we decide to break the server roles the way we did and not in other ways?
  • We wish to support a full cluster setup and maintain a modular structure to allow for easier server roles and respected maintenance and updates.
    The Kaltura architecture allows designating each server (or group of servers for more computing power and redundancy purposes) with its own dedicated role (batch, front, Sphinx, etc.) but it is also possible to have a server that has multiple or all roles assigned to it.
    By creating a package for each server role, we allow for easy support of both setup types, and gain a modular structure that allows for simpler updating/patching each individual server role.
    If you wish to assign only one role, install the package for that role (all packages will also automatically require the kaltura-common package to be installed by the Linux package manager), if you wish to have an all-in-one server, simply install all packages.
• • Which official packages are being used and which are you bundling specifically?
  • This project aims to support complete self-supported local installs of Kaltura on a lean environment, downloading the least possible number of packages and requiring the least possible number of additional repositories.
    In addition, the packages project aims to maintain the perfect version compatibility across the various platform components.
    To meet these requirements, the packaging project will only depend on official base distribution packages that are widely used and known to be stable and accepted.
• • Are we compiling the requirements (e.g. sphinx, ffmpeg) or relying on other applications already distributed in the official repositories?
  • For MySQL and PHP, we use standard packages from the distributions official repository.
    For Sphinx and ffmpeg, this cannot be done because a robust and stable Kaltura deployment requires very particular versions, often because they are newer and include fixes for crucial bugs, and additional functionality.
    Therefore, some libraries will be supplied as separate packages which the Kaltura packages will depend on.
    Note that these packages will be installed under the Kaltura directory structure and thus will not conflict or override the parallel packages from the official distribution repository.
    Nevertheless, this is a good time to remind that video processing is a demanding task and should be on its own dedicated machine - it is not recommended to install Kaltura alongside other production software on the same machine.
• • What are the usernames, groups and permissions each package configures, and why do we need them?
  • The Kaltura platform basically works with the following users:
    
    • Apache user: the default one each distribution sets up - for Debian/Ubuntu it is ‘www-data’, for RHEL/CentOS/FC it is ‘apache’.
    • The ‘kaltura’ user: used for running our Sphinx daemon, the populate daemon (responsible for syncing MySQL and Sphinx) and various other aiding scripts, like cache cleaning, etc.
    For security purposes, none of the Kaltura processes will run as root.
    In addition, the post-install scripts will provide an option of specifically configuring usernames and groups in a non-standard environment deployments.
• • Will internet (www) connection be required to deploy Kaltura?
  • Having public internet connectivity is not essential for installation (though does make things easier) - One could download the RPM/deb files and manually install, offline, each package in its turn according to the desired server role. There are also meta-packages that will deploy the needed packages automatically from the local copy of the package files.
    
  Additionally, this project exposes the complete repository code (and even a chroot env for developers). One could also deploy a complete clone of the packages repository in the local network and point local machines to it.
• • How will repurposing / changing server roles be handled?
  • Re-deploying server roles (e.g. deciding that a batch server will now become a front server instead) is easily handled when using packaged installs, and greatly due to the modular architecture chosen.
    By having a package for each server role, reconfiguring an existing server to operate as another (or additional) role, will be as simple as unintstalling the existing packages, and installing the packages of the desired role/s. The post-install script of the newly installed packages will take care of the new configurations required for each desired server role.
• • How will packages help simplify the upgrade process of Kaltura servers?
  • By leveraging standard package managers, we plan to achieve a faster, incremental release cycles, with clear change-log and native system package versions management.
    
    • Faster Incremental Releases - By having a clearly defined mechanism for releasing packages, we will also be able to release more often, incrementally resulting in smaller and easier to manage upgrade flows.
    • Version Controlled - A key benefit of leveraging standard package managers is having clear change-logs and robust package version management. Package managers will know which version your system is on, and what upgrade scripts should be run to upgrade to a desired new version.
    • PreInstall, PostInstall, PreUnInstall and PostUnInstall hooks - These native package manager system hooks allow us to clearly define scripts that can verify config-files diffs, perform cleanup tasks and more.
    For example, handling of modified local config files while upgrading to a new config file format - Such scenario will be handled by the package manager, first the package manager will identify that there was a change in the config files in the version being deployed, a new template file will be deployed, and when the post-install script will run, it will extract the values from the local config file and migrate the values into the new template file, keeping a backup of the old config file.
• • How could I automate installation of packages?
  • Using "answers file" (example of an answers file for illustration), package managers can be made to perform installations in a silent-mode, running complete deployment including post-install configurations.
    
• • Any plans to automate deployment and scaling of Kaltura using Chef or Puppet?
  • As part of Phase C.2, we published Deploying Kaltura using Opscode Chef, and we also plan to release the recipes on the Chef cookbooks site.
    If you are experienced with other technologies like Puppet, The Foreman, Cloudify or other automation / cloud management tools - we would love your help! :)
• • How will you make cluster / multi-server environment deployments easier?
  • Multi-server environment deployments are a central driving consideration for this project, there is a significant effort to define and separate packages based on server roles and components. By having multiple packages that are self-contained and a clear chain of dependency management (which is handled by the system package manager), deploying across the network is made much simpler.
    There are multitude of tools that can be leveraged to automate deploying Kaltura servers across the network and even in rules-based automation of scaling or redundancy scenarios. Depending on the environment restrictions and business constraints, these tools include the aforementioned planned Chef scripts, and other common tools such as RedHat Satellite, Spacewalk and others.
    If you've got experience with automation, and would like to suggest best practices, ideas or concerns - please join us at the weekly IRC (Mondays at 10am EST) or post to the issue queue.
• • How are system specific tuning configurations handled?
  • For example: in a Batch server role some configurations are dependent on how much RAM and CPU resources are available.
    CPU and RAM based tuning configurations will eventually be performed automatically by the post-install script, which will calculate the ideal values based on the system available resources.
• • How can this handle non-standard install directories (i.e. installing outside /opt/kaltura)?
  • Although not ideal, and highly discouraged, non-standard deployments could be handled in either of the following ways:
    • Using a chroot under /opt/kaltura and have the chroot itself reside anywhere else.
    • Install under /opt/kaltura and post installation move everything to different path (note that you will have to manually modify configuration files), Or move to a different path and keep a symlink to /opt/kaltura. This option makes upgrades a messy process.
    • Clone the code repository, modify the /opt/kaltura path prefix, deploy a local version of the packages repository, and install using the local repository. Note that this will require keeping up to date with new releases manually and re-deploying each new release to the local repository.
• • I've installed Kaltura. How can I tell if there are any errors?
  • You can use the following tail and grep combination to monitor the logs for errors:
    tail -f /opt/kaltura/log/*.log | grep "ERR:\|PHP\|trace\|CRIT" --color=always
• • What is Kaltura?
  • Kaltura is the world's first Open Source Online Video Platform, transforming the way people work, learn, and entertain using online video. The Kaltura platform empowers media applications with advanced video management, publishing, and monetization tools that increase their reach and monetization and simplify their video operations. Kaltura improves productivity and interaction among millions of employees by providing enterprises powerful online video tools for boosting internal knowledge sharing, training, and collaboration, and for more effective marketing, sales and customer support. Kaltura offers next generation learning for millions of students and teachers by providing educational institutions disruptive online video solutions for improved teaching, learning, and increased engagement across campuses and beyond. Kaltura also provides operators and service providers with a Cloud TV platform that enables them to launch next generation TV services to their audience. For more information visit: http://corp.kaltura.com, http://www.kaltura.org and http://www.html5video.org.