Info | |||||||
---|---|---|---|---|---|---|---|
Contents:
Related pages:
|
Kiuwan On-Premises Installation Guide
System Architecture Overview
Kiuwan On-Premises (KOP) is based on a client-server architecture with the following major server components:
...
KOP also supports distributed architecture with high-availability and load-balancing. For these scenarios, please contact Kiuwan support.
Server Host Requirements
Installation requirements
Kiuwan On-Premises (KOP) solution is deployed on a centralized server host (all the components are installed in the same host) and accessed by users via HTTP or by IDE plugins (Eclipse, Visual Studio).
...
Info | ||
---|---|---|
| ||
|
...
- Recommended size for KOP persistent volume is 50 Gb
- Try to specify an independent disk/partition, so you can easily increase it later if needed
Special requirements for RedHat, CentOS and Fedora
Info |
---|
If your docker server is running on RedHat, CentOS or Fedora, be sure the filesystem where docker is installed supports d_type. If d_type isn't supported, you've to change it as soon as possible to prevent strange filesystem issues with Docker containers. |
Some installation of above operating systems aren't configured with d_type support (see http://www.pimwiddershoven.nl/entry/docker-on-centos-7-machine-with-xfs-filesystem-can-cause-trouble-when-d-type-is-not-supported)
Running on XFS without d_type support causes Docker to skip the attempt to use the overlay or overlay2 driver.
Existing installs will continue to run, but produce an error. See https://docs.docker.com/storage/storagedriver/overlayfs-driver/#prerequisites
You can check if your existing XFS filesystem has d_type enabled by running these commands
Code Block | ||
---|---|---|
| ||
$ docker info | grep "Supports d_type:" Supports d_type: true $ xfs_info /docker-mount-point | grep ftype naming =version 2 bsize=4096 ascii-ci=0 ftype=1 |
In case you get d_type: false or ftype=0 , you have to create a new XFS filesystem with d_type support enabled.
Unfortunately, it isn't possible to enable d_type support on an existing filesystem.
You have two options:
- either you add a new disk and you create a new XFS partition on it, or
- you've to backup your existing data and recreate the XFS filesystem with d_type support enabled.
Creating a new XFS filesystem with d_type enabled is as easy at the following command:
Code Block | ||
---|---|---|
| ||
$ mkfs.xfs -n ftype=1 /mount-point |
CPU and memory requirements
Server CPU and memory requirements depend on several factors:
...
Info |
---|
It's important to notice that we're assuming the typical use case where code analyses are performed locally through Kiuwan Local Analyzer Deprecated, and therefore out of server machine (tipically at a user machine or within a build CI system). Therefore, when we talk about analysis we mean 2nd phase of the analysis (i.e. indicator calculation that is executed within the host server) |
...
- In case you need a higher number of parallel analyses executions, you should add 1 CPU core and 512 Mb for any additional required parallel analysis
- In case you need to improve response time to web users, conside to add 1Gb for every 50 additional web users.
Dependencies
KOP docker image is build based on these external dependencies
Software | Version |
---|---|
Ubuntu | 16.04 |
MySQL | 5.6 (latest available build) |
WildFly | 10.0.0.Final |
Redis | 3.0.7-1 |
JDK | 8.0_202 |
Ant | 1.7.1 |
Gradle | 4.7 |
AspectJ | 1.9.1 |
NPM | 10.7.0 |
MongoDB | 3.6.5 |
MySQL JConnector | 5.1.39 |
Additional Ubuntu packages | apparmor-utils \ ca-certificates \ apt-transport-https \ |
Installation steps
Before installing KOP, make sure that you understand the System Architecture Overview, that your server host(s) complies with the Server Host Requirements, and that you have properly prepared the Environment as follows:
Info | ||||
---|---|---|---|---|
Before installing KOP, make sure your server host complies with the Server Host Requirements
Installation Steps:
|
Step 1. Unpackage KOP Installation Package
Info |
---|
KOP Installation Package consists on a tarball gz file (docker-for-kiuwan.tar.gz) containing all the kiuwan docker files.
KOP Licenses are distributed through a zip file that contains two license distribution zipfiles:
To install the licences:
Please visit How to Install KOP Licenses is you need to reinstall (or upgrade) licenses after the docker container is built. |
Step 2. Configure Kiuwan Docker image
Info |
---|
After unpackaging, you should configure the docker image to be built. Configuration of Kiuwan docker image is currently done by creating a configuration file (settings.custom) that will be used during the image building process. There's a file (settings.tpl) you can use as a template to configure your own settings.
|
KOP Deployment scenarios
Before to install KOP, you should decide your KOP deployment scenario.
...
Below picture displays these different scenarios
Scenario 1 - Direct connection to KOP server
First, you should decide the external URL that KOP clients will use. By KOP clients, we mean Browsers, Kiuwan Local Analyzer, Kiuwan for Developers and API-REST clients.
...
To configure your KOP in this scenario, please follow next steps
Scenario 1.1 - Direct KOP access through HTTP
Your URL will be something like : http://mykop.mydomain.com:7080/saas
...
PropertyName | Value | Comment |
---|---|---|
KIUWAN_HOST KIUWAN_PORT | mykop.mydomain.com | Hostname (Fully Qualified Domain Name, see note below ) and port number to access KOP. These variables will be used to build Kiuwan URL. Example (with default values):
Fully Qualified Domain Name (FQDN): https://en.wikipedia.org/wiki/Fully_qualified_domain_name
Must match KIUWAN_WEBSERVER_HOST |
KIUWAN_WEBSERVER_PROTOCOL | http | |
KIUWAN_WEBSERVER_HOST | mykop.mydomain.com | Must match KIUWAN_HOST |
Scenario 1.2 - Direct KOP access through HTTPS
Your URL will be something like : https://mykop.mydomain.com:7080/saas
...
First, create the image as HTTP using 1.1 HTTP scenario configuration (Kiuwan On-Premises Installation GuidePremisesInstallationGuide-Scenario1.1-DirectKOPaccessthroughHTTP)
Once the image is built, convert it to HTTPS following steps described at Kiuwan On-Premises Installation GuidePremisesInstallationGuide-Step5.(Optional)MakingKiuwanOnPremiserunningonHTTPS
Scenario 2 - Connection to KOP server through a WebServer
In this scenario, a webserver (typically Apache) is used as a frontend server to redirect to KOP appserver.
As in the “direct” scenario, you should establish the URL to be used to connect to KOP.
Let suppose that will be something like http://mykop.mydomain.com:80/saas
Additionally, you will need to configure the “internal” KOP app server URL, for example : http://my_internal_kop.mydomain.com:7080/saas
Please note that WebServer can communicate to KOP appserver either through HTTP or AJP. You can configure the corresponding ports (KIUWAN_PORT and KIUWAN_APPSERVER_AJP_PORT).
PropertyName | Value | Comments |
---|---|---|
HTTPS | [ ON | OFF ] | |
KIUWAN_WEBSERVER_PROTOCOL | [ http | https ] | If HTTPS=OFF, specify http IF HTTPS=ON, specify https |
KIUWAN_WEBSERVER_HOST KIUWAN_WEBSERVER_PORT | mykop.mydomain.com | External KOP hostname (Fully Qualified Domain Name, see note below ) and port number to access KOP. These variables will be used to build Kiuwan URL. Example (with default values):
Fully Qualified Domain Name (FQDN): https://en.wikipedia.org/wiki/Fully_qualified_domain_name |
KIUWAN_HOST KIUWAN_PORT | my_internal_kop.mydomain.com | Internal KOP hostname (Fully Qualified Domain Name, see note below ) and port number to access KOP. These variables will be used to build internal Kiuwan URL. Example (with default values):
Fully Qualified Domain Name (FQDN): https://en.wikipedia.org/wiki/Fully_qualified_domain_name
KIUWAN_HOST must be different to KIUWAN_WEBSERVER_HOST. KIUWAN_PORT must be different to KIUWAN_WEBSERVER_PORT only if the frontend server and KOP appserver execute in the same machine. |
KIUWAN_APPSERVER_AJP_PORT | defaults to 7009 | Port that the front-end server will use to communicate with KOP appserver through AJP |
KOP Configuration properties
PropertyName | Default value | Meaning |
---|---|---|
SysAdmin configuration | ||
KIUWAN_DOCKER_IMAGE | Name of the docker image (repository name) to be built | |
DIR_PERSISTENT_VOLUME | /Datavol | Root directory of docker server where the container volume will be bound |
KIUWAN_VOL | /kiuwan_vol | Directory of the container where KOP will be installed. |
EXTERNAL_PORT_TO_MAP_MYSQL_INTERNAL_PORT | 3306 | Docker server port to access container's MySQL
|
EXTERNAL_PORT_TO_MAP_SSH_PORT | 2222 | Docker server port to access container thorugh SSH
|
MONGODB_START | false | Depending of your Kiuwan On Premises purchased license:
|
Proxy settings | ||
USE_PROXY | false |
|
HOST_PROXY PORT_PROXY | Hostname (or ip address) and port number of proxy used for Intenet connectivity during the image building process | |
USER_PROXY PWD_PROXY | Username and password to connect Proxy.
| |
NO_PROXY | localhost,127.* | Comma-separated list of destinations that will not be accessed through proxy. Do not use quotes (single or double) to enclose these values. Important: KIUWAN_HOST value is automatically appended |
KOP users | ||
KIUWAN_USER KIUWAN_GROUP | kiuwan kiuwan | System user/group for KOP installation and execution |
MYSQL_ROOT_PWD | root | Password for root user of KOP's MySQL instance |
KIUWAN_MYSQL_USER KIUWAN_MYSQL_USERPWD | kiuwan kiuwan | Username and password used by KOP to access MySQL database |
SSH_ROOT_PWD | password | Password for root user (SSH) of Kiuwan Container |
KIUWAN_EMAIL_ADMIN_ACCOUNT | Email address of kiuwanadmin user
| |
KOP e-mail (SMTP) configuration | ||
KIUWAN_SMTP_HOST | SMTP host and port | |
KIUWAN_SMTP_USER_ACCOUNT KIUWAN_SMTP_USER_PASSWORD | SMTP account : username and password | |
KIUWAN_EMAIL_USER_ACCOUNT | Sender email address for KOP email notifications | |
Advanced configuration (WildFly, MySQL, Redis, etc) | ||
INNODB_BUFFER_POOL_SIZE | 2G | |
APPS_JVMMAXMEMORY | 3G | Max ammount of heap memory to be used by WidlFly |
KIUWAN_REDIS_MAXMEMORY | 2gb | Max memory limit to be used by Redis server (https://redis.io/topics/config) |
You can visit Advanced Configuration for additional configuration parameters.
Step 3. Build the Kiuwan Docker image
Info |
---|
After configuration, you will build the Kiuwan docker image.
IMPORTANT: Please be sure you have already installed KOP licenses, otherwise the docker image will not be built !!
To build the docker image: cd $KOP_INSTDIR/master.YYYYMMDDHHMM.buildnumber ./create_and_init_kop.sh
This step requires Internet connectivity. If your installation uses an proxy, you should have previolsy configured the proxy settings.
In case you need to execute the Kiuwan container in a host server without Internet connectivity:
|
...
After image creation, you will be able to list the images in your docker server and identify your kiuwan image by issuing the next command:
docker images
Step 3.1 (Optional) Moving the image to another server host
After creating the image, if you want to move it to another host server you first must save it to a tarball by executing:
...
docker load --input kop_v12_saved_image.tar
Step 4. Executing the Kiuwan Docker container
Info |
---|
After executing docker image build script (./create_and_init_kop.sh ), two scripts are generated based on settings.custom configuration file
run_kop.sh Script to start the Kiuwan docker container and make KOP fully available and accessible to users.
run_init_kop.sh Script to initilize the Kiuwan docker container (i.e. resetting completely the KOP installation to factory defaults) without to create a new image. This script will remove any existing data of KOP installation (for example, all the database contents will be deleted).
IMPORTANT:
You can stop the Kiuwan container by issuing the following command: docker stop <my_container_name> |
Step 4.1 Execution in debug mode (only sshd)
Just in case you are running with problems while executing Kiuwan container, you can run it in debug mode.
...
Info |
---|
After the container is built, you can execute (run) the Kiuwan container in debug mode by issuing the following command:
docker run --rm --name <my_container_name> \ -h <my_container_host_name> \ -v <server_host_mount_dir:container_mount_dir> \ -p <ssh_port_ext>:22 \ -d \ <image_name:version> |
Step 5. (Optional) Making Kiuwan On Premise running on HTTPS
In you need to execute Kiwuan On Premise over HTTPS protocol, please follow next steps.
Step 5.1 Create your KOP SSL configuration file
SSL configuration is currently done by creating a configuration file (ssl.custom) within ssl directory
...
Property Name | Default value | Meaning |
---|---|---|
Commons | ||
SSL_O | Your Organization | |
SSL_LOCALITY | Your Locality | |
SSL_STATE | Your State | |
SSL_COUNTRY | Your Country | |
SSL_OU | Your Organization Unit | |
Keystore | ||
SSL_KS_PWD | Password for Keystore that will be created | |
Aliases | ||
SSL_ALIAS | wildfly | Alias of the Certificate to be created. |
HTTPS_PORT | 8443 | HTTPS port |
Step 5.2 Create the Private key and Certificate Signing Request (CSR) for your server
- within ssl directory execute the script CreateKey_and_ReqCSR.sh
- this script generates the CSR file under ssl/certs
- that file is named <yourhost.yourdomain.com>.csr , according to $KIUWAN_HOST configuration property
- send CSR file to your CA (Certificate Authority)
- CA will send back to you two files:
- CA's Certificate file (IMPORTANT: rename it to ca.crt )
- your host's Certificate file : for example yourhost.yourdomain.com.crt
- copy received files to ssl/certs directory
Step 5.3 Create the Keystore and switch from HTTP to HTTPS
- within ssl directory execute the script TransferFilesToContainer.sh
- this script transfers your server's certificate, your private key and CA's certificate to KOP container
also, it transfers the script templates that will be used to create the keystore and to change the configuration from http to https
within ssl directory execute the script run_create_Keystore.sh
- this script executes (into the container) the script create_Keystore.sh (created from template create_Keystore.tpl)
- within ssl directory execute the script run_change_ToHTTPS.sh
- this script stops wildfly service and executes the script change_ToHTTPS.sh (created from template change_ToHTTPS.tpl), this script will create files with .rollback extension of modified ones
- then, it starts wildfly service
Info |
---|
After the conversion to HTTPS, please download a new version of Kiuwan Local Analyzer from your KOP site. |
Step 5.4 Just in case you are using your own CA, make it valid to your browsers and Java
If the certificate is signed by your own Certification Authority, the browsers will not recognize it as a valid CA and you will get an error messages such as:
...
- docker commit <kop docker name> <kop docker image>
- NOTE: run 'docker ps' to get NAMES and IMAGE values of your KOP container
Rollback to HTTP
...
After execution, KOP will come back to configuration previous to the execution of run_change_ToHTTPS.sh script.
Step 6. Accessing Kiuwan On Premise
KOP URL
Once Kiuwan On Premise container is running, you can access it from a browser in the following URL:
http[s]://<KIUWAN_HOST>:<KIUWAN_PORT>/saas
where KIUWAN_HOST and KIUWAN_PORT match the values of those properties as configured in settings.custom
KOP REST-API URL
http[s]://<KIUWAN_HOST>:<KIUWAN_PORT>/saas/rest/v1
KOP built-in users
KOP comes with the following built-in users.
- kiuwanadmin (password: kiuwanadmin)
- access to Kiuwan "functional" administration modules such as Users, Applications and Model Management (see Admin Guide )
- log in as kiuwanadmin to create users of your KOP instance
- sysadmin (password: sysadmin)
- log in as sysadmin to access functionalities related to monitoring and tuning KOP execution
- sysadmin will give you access KOP sysconsole
Upgrading your KOP installation
In order to maintain your KOP installation up-to-date to new features, bugfixes, etc, you will have to upgrade your KOP installation.
Log in to Console as SysAdmin and check the status of your KOP installation:
The following pages describe the upgrading Installation procedures of releases of KOP.
Before starting the upgrade process, it's always advisable to make a backup of current KOP installation.
- stop the container
- docker stop <$KIUWAN_HOST>
- make a backup of MySQL (optional, step #3 also makes a backup copy of MySQL)
- from docker server, make a copy of permanent volume ( $DIR_PERSISTENT_VOLUME setting ).
- cp -rpf <$DIR_PERSISTENT_VOLUME/$KIUWAN_HOST> <$DIR_PERSISTENT_VOLUME/$KIUWAN_HOST>.bck
- Substitute DIR_PERSISTENT_VOLUME and KIUWAN_HOST with values configured at settings.custom file.
- start the container
- run_kop.sh
Basically, a new relase is distributed as a new KOP Installation Package, consisting on a tarball gz file (docker-for-kiuwan.gz) containing all the new kiuwan docker files.
- copy the new distribution tarball to your host server’s updates directory ($KOP_INSTDIR/updates/download)
- uncompress the distribution tarball
- tar xvzf docker-for-kiuwan.tar.gz
- a $KOP_INSTDIR/updates/download/master.YYYYMMDDHHMM.buildnumber directory will be created
- change directory to $KOP_INSTDIR/updates/bin
To upgrade Kiuwan Core AND Kiuwan Clients:
- cd to $KOP_INSTDIR/updates/bin
- execute the script ./updateKiuwanLauncher.sh
- this script makes all the work to upgrade your current KOP app instance to the new one (CORE and CLIENTES will be updated)
To upgrade ONLY Kiuwan Clients:
- cd to $KOP_INSTDIR/updates/bin
- execute the script ./updateClientsLauncher.sh
- this script makes new KLA and K4D ready to be downloadable by KOP client users
Advanced Configuration
As above specified, settings.custom file allows you to specify most (but not all) configuration parameters.
...
Component | PropertyName (=default value) | Meaning |
---|---|---|
TimeZone | KIUWAN_TIMEZONE='UTC' | Timezone to be used by Kiuwan application. |
WildFly | APPS_JVMMINMEMORY=512M | Max and Min ammount of heap memory to be used by WidlFly. |
APPS_JVMMINMETASIZE=96M APPS_JVMMAXMETASIZE=512M | Max and min ammount of perm gen memory to be used by WildFly | |
MySQL | innodb_buffer_pool_size=2G | Default MySQL parameters used to create database instance. (see https://dev.mysql.com/doc/refman/5.6/en/ for reference) redis |
Redis | KIUWAN_REDIS_MAXCLIENTS=100 | Maximum number of Redis clients that could be handled simultaneously (https://redis.io/topics/clients) Max memory limit to be used by Redis server (https://redis.io/topics/config) |
Appendix 1 - Configuring Apache as FrontEnd WebServer to KOP
If you choose Apache web server as the front-end for your Kiuwan on premise installation, you may follow the following configuration examples and/or check your current configuration agrees with the suggested examples.
HTTP Protocol
For this kind of access you just need to add a virtual host and configure how Apache will talk to Kiuwan.
...
Code Block | ||
---|---|---|
| ||
<VirtualHost *:[KIUWAN_WEBSERVER_PORT]> ServerName [KIUWAN_WEBSERVER_HOST] ServerAdmin webmaster@localhost DocumentRoot /var/www/html LogLevel debug ErrorLog ${APACHE_LOG_DIR}/error.log CustomLog ${APACHE_LOG_DIR}/access.log combined ProxyRequests Off ProxyPreserveHost On <Proxy *> Require all granted </Proxy> ProxyPass / [ajp|http]://[KIUWAN_HOST]:[KIUWAN_PORT]/ ProxyPassReverse / [ajp|http]://[KIUWAN_HOST]:[KIUWAN_PORT]/ <Location /> Require all granted </Location> </VirtualHost> |
HTTPS Protocol
For this kind of access you will need:
...
[KIUWAN_WEBSERVER_HOST].crt: the certificate for your kiuwan host.
[KIUWAN_WEBSERVER_HOST].key: the private key.
ca.crt: the trusted CA certificate.
Refer to Kiuwan On-Premises Installation Guide PremisesInstallationGuide-Step5.2CreatethePrivatekeyandCertificateSigningRequest(CSR)foryourserver for more information on how to create and install certificates.
...