How to create private registry and configure Apphost for serving apps while installing

Development environment side: CentOS 8
--
okanx@control-plane ~]$ cat /etc/docker/daemon.json
{
"insecure-registries" : ["172.16.60.128:5000"]
}


okanx@control-plane ~]$ cat /opt/docker-registry/docker-compose.yml
version: '3'

services:
  registry:
    image: registry:2
    ports:
    - "5000:5000"
    environment:
      REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data
    volumes:
      - ./data:/data

[aokanx@control-plane ~]$ cd /opt/docker-registry/
[aokanx@control-plane docker-registry]$ docker-compose up



Apphost side:
--

[root@apphost ~]# cat /etc/rancher/k3s/registries.yaml
mirrors:
  "172.16.60.128:5000":
    endpoint:
      - "http://172.16.60.128:5000"
[root@apphost ~]# manageAppHost registry --registry=172.16.60.128:5000

QRadar SOAR (Resilient): Expired K3s certificates are not automatically rotated causing connection issues on Apphost

Problem

Cached K3s certificates are not cleared when automatically rotated.

K3s generates internal certificates with a 1-year lifetime. Restarting the K3s service automatically rotates certificates that expired or are due to expire within 90 days. However, the version of K3s used with App Host does not clear out the cached certificate, which causes the same problem. Therefore, the cache needs to be cleared manually.

Symptom

Using the kubectl cli tool results in the following error:

 Unable to connect to the server: x509: certificate has expired or is not yet valid

***Note: Apps that are currently running continue to run with no issues

Cause

The currently used version of K3s (v1.18) does not clear the cached certificate.  So even if the certificates are rotated by a K3s restart, the problem persists.

Diagnosing The Problem

Changes to an existing app that uses the SOAR platform does not succeed. This includes file changes, secret changes, deploy or undeploy, and upgrade requests.

Installing a new app, the status remains in a ‘Deploying’ state. A tooltip instructs the user to run sudo appHostPackageLogs. Running this command or any other command that starts with kubectl gives you the following error:

#: kubectl get pods -A
Unable to connect to the server: x509: certificate has expired or is not yet valid

You can check the expiration data of a cached certificate by running the following command on the App Host server:

openssl s_client -connect localhost:6443 -showcerts < /dev/null 2>&1 | openssl x509 -noout -enddate 

Resolving The Problem

As a precautionary measure backup the TLS dir.

sudo tar -czvf /var/lib/rancher/k3s/server/apphost-cert.tar.gz /var/lib/rancher/k3s/server/tls

Remove the following file.

sudo rm /var/lib/rancher/k3s/server/tls/dynamic-cert.json

Remove the cached certificate from a kubernetes secret.

sudo kubectl --insecure-skip-tls-verify=true delete secret -n kube-system k3s-serving

Restart the K3s service to rotate the certificates.

sudo systemctl restart k3s

Verify that kubectl commands function.

sudo kubectl get pods -A

Additionally, you can verify that all K3s internal certificates are no longer due to expire.

sudo su

for i in `ls /var/lib/rancher/k3s/server/tls/*.crt`; do echo $i; openssl x509 -enddate -noout -in $i; done

Or run

curl -v -k https://localhost:6443 [https://localhost:6443] to confirm the new date of your app host cert

 

 

Reference:

https://supportcontent.ibm.com/support/pages/expired-k3s-certificates-are-not-automatically-rotated-causing-connection-issues

Utility Functions for Resilient (fn_utilities-1.0.6.zip) installation problem

Resilient Functions simplify development of integrations by wrapping each external activity into an individual workflow component. These components can be easily installed, then used and combined in Resilient workflows. The Resilient platform sends data to the function component that performs an activity then returns the results to the workflow. The results can be acted upon by scripts, rules, and workflow decision points to dynamically orchestrate the security incident response activities.

The Utility Functions integration package contains several useful workflow functions for common automation and integration activities in Resilient. These include:

    Function to call generic REST/JSON web service APIs,
    Function to run arbitrary shell scripts (bash and PowerShell),
    Functions to fetch SSL certificates from a server and parse them,
    Functions to work with Excel, HTML, XML, JSON and EML files,
    Functions to work with Resilient attachments: calculate hashes, list and extract ZIP archives, convert to and from base64
    And more.



Problem:

[root@resilient ~]# pip install fn_utilities-1.0.6.zip
Processing ./fn_utilities-1.0.6.zip
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    IOError: [Errno 2] No such file or directory: '/tmp/pip-req-build-k_Apec/setup.py'
   
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-k_Apec/

Solution:

[root@resilient ~]# unzip fn_utilities-1.0.6.zip
Archive:  fn_utilities-1.0.6.zip
  inflating: fn_utilities-1.0.6.tar.gz 

[root@resilient ~]# pip install fn_utilities-1.0.6.tar.gz