Friday, December 27, 2019

Is the Password to the Private Key Correct?

When we generate a public-private key pair for public key cryptography, such as, RSA, we can use a password to control access to the private key. We would know if it is the case by viewing the content of the key file, e.g.,

$ sudo head -1 myprivatekey.key
-----BEGIN ENCRYPTED PRIVATE KEY-----

The problem is that I don't know which password is correct because I have a few. If we are using openssh, we can easily verify if a password is correct by using ssh-keygen with the -y option -- the manual states,

-y      This option will read a private OpenSSH format file and print an
        OpenSSH public key to stdout.

Knowing this, we verify whether a password is correct or not by
ssh-keygen -y -f ./myprivatekey.key; echo "exit code is " $?
Enter passphrase: xxxxxxxx
Load key "./myprivatekey.key": incorrect passphrase supplied to decrypt private key
exit code is  255

which shows that the password I entered was incorrect. However, we entered a correct one, we would observe,

ssh-keygen -y -f ./myprivatekey.key; echo "exit code is " $?
Enter passphrase: yyyyyyyy
exit code is  0

Thursday, December 5, 2019

On a Debian-based Linux Distribution, Which Package Has a Given File?

We often wonder which package we should install to use a command. On Debian-based distributions, we can use apt-file to locate file that in packages that we have or haven't installed, and use dlocate to locate a file in packages that have been installed.

For example, to determine which package the ip command is in, we can run the following.

First, make sure we have installed apt-file

sudo apt-get install apt-file

Second, make sure we have packages indexed and up-to-date, be it installed or not.

sudo apt-file update

We can now determine which package has the ip command,


apt-file search ip

The result may be too long to be useful. To shorten the result, we apply heuristics. We know that the ip is a command, and it is probably in a bin directory, the name of the file should be ip,

apt-file search ip | grep "bin" | grep "/ip$"

Now the output is

iproute2: /bin/ip
iproute2: /sbin/ip

If we know that we have had the iproute2 installed, we can use dlocate as follows.
First, we make sure that we have installed dlocate.

sudo apt-get install dlocate

Next, we run dlocate with some heuristics as in,

dlocate ip | grep "bin" | grep "/ip$"

The output is,

iproute2: /bin/ip
iproute2: /sbin/ip

Wednesday, November 13, 2019

When SFC Cannot Fix Corrupted Windows Files, DSIM Comes to Rescue

Just as the title suggested, on a Windows 10 host, I encountered a case that SFC reported that it could not fix corrupted files, and DSIM in this case helped. Below is the screen transcript.

C:\WINDOWS\system32>sfc /scannow

Beginning system scan.  This process will take some time.

Beginning verification phase of system scan.
Verification 100% complete.

Windows Resource Protection found corrupt files but was unable to fix some of them.
For online repairs, details are included in the CBS log file located at
windir\Logs\CBS\CBS.log. For example C:\Windows\Logs\CBS\CBS.log. For offline
repairs, details are included in the log file provided by the /OFFLOGFILE flag.

C:\WINDOWS\system32>Dism /Online /Cleanup-Image /RestoreHealth

Deployment Image Servicing and Management tool
Version: 10.0.17763.771

Image Version: 10.0.17763.805

[==========================100.0%==========================] The restore operation completed successfully.
The operation completed successfully.

C:\WINDOWS\system32>sfc /scannow

Beginning system scan.  This process will take some time.

Beginning verification phase of system scan.
Verification 100% complete.

Windows Resource Protection found corrupt files and successfully repaired them.
For online repairs, details are included in the CBS log file located at
windir\Logs\CBS\CBS.log. For example C:\Windows\Logs\CBS\CBS.log. For offline
repairs, details are included in the log file provided by the /OFFLOGFILE flag.

C:\WINDOWS\system32>sfc /scannow

Beginning system scan.  This process will take some time.

Beginning verification phase of system scan.
Verification 100% complete.

Windows Resource Protection did not find any integrity violations.

C:\WINDOWS\system32>



Thursday, October 24, 2019

LOG4J on Windows Complaining about Console Code Page

I was running the Windows JVM with LOG4J 2.8.1 with SLF4J 2.7.5 on the Windows Subsystem for Linux and encountered an error as follows,

2019-10-24 16:03:19,902 main ERROR Unable to inject fields into builder class 
for plugin type class org.apache.logging.log4j.core.appender.ConsoleAppender, 
element Console. java.nio.charset.UnsupportedCharsetException: cp0

In the Windows Subsystem for Linux terminal, the code page returned was 0, which was the source of the code page cp0.

user@WSL:/mnt/c/Windows/System32$ chcp.com
Active code page: 0
user@WSL:/mnt/c/Windows/System32$

Interestingly, the returned code page is on Windows Command Prompt,

C:\Windows\System32>chcp
Active code page: 437

C:\Windows\System32>

To get rid of the problem, we need to inform Java the encoding scheme, e.g.,

java -Dsun.stdout.encoding=UTF-8 -Dsun.err.encoding=UTF-8 FooClass

where we assume we run the FooClass class.

Tuesday, October 22, 2019

Using tuned to Configure PostgreSQL Database Management System

Tuned is a profile-based system tuning tool for dynamic and static tuning of system settings. I find it very useful even if for configuring system settings. Tuned comes from recent release of Fedora Linux distribution. The following steps are just to make sure that we have the tuned package in the system, and start and enable the tuned service.

sudo dnf install tuned
sudo systemctl start tuned
sudo systemctl enable tuned

Below are a few steps to show how we use tuned to manually adjust system settings (or static-tuning):

  • Listing existing profiles
    
    tuned-adm list
    

  • To activate a profile or to reactivate a profile after you have revised it,
    
    sudo tuned-adm active postgresql
    

    In this example, we have created a profile of our own to tune the system for running PostgreSQL Database Management System.
To create such a profile, follow the steps:
  1. Create a directory at /etc/tuned, e.g., if we wish to have a profile called postgresql, we create the directory postgresql at /etc/tuned,
    
    sudo mkdir /etc/tuned/postgresql
    

  2.  Create a profile at the directory you created, e.g., create tuned.conf at /etc/tuned/postgresql. The following is an example profile for PostgreSQL
    
    $ cat /etc/tuned/postgresql/tuned.conf
    #
    # tuned configuration
    #
    
    [main]
    summary=Optimize for running PostgreSQL inside a virtual guest
    
    [cpu]
    governor=performance
    energy_perf_bias=performance
    min_perf_pct=100
    
    [disk]
    readahead=>4096
    
    [sysctl]
    vm.dirty_ratio = 2
    vm.dirty_background_ratio=1
    vm.overcommit_memory=2
    vm.swappiness = 0
    vm.nr_hugepages=1192
    
    [vm]
    transparent_hugepages=never
    
    

    This profile basically sets the system to use 2MB hugepages for PostgreSQL. It is worth nothing that in the PostgreSQL's $PGDATA/postgresql.conf configuration file, we have these lines:
    
    shared_buffers = 2GB
    huge_pages = on
    

    Since shared buffers is 2GB/(hugepage size) = 2GB/2MB = 1024 pages, the vm.nr_hugepages (the number of reserved hugepages) cannot be less than 1024 pages plus additional pages for the process itself and others; otherwise, PostgreSQL will fail to start.

Friday, October 18, 2019

Enabling pg_stat_statements for PostgreSQL Database System Tuning and Analysis

To facilitate PostgreSQL database system performance tuning and analysis, we ought to collect query statistics, such as, the amount of time that a query takes. For this, we should enable pg_stat_statements extension. The following shows the steps on a Fedora Linux system:

  1. Install the postgresql-contrib package. In the system shell, run command

    sudo yum install postgresql-contrib

  2. Create pg_stat_statements extension for the database whose query performance we wish to analyze, e.g., we wish to tune performance for database template1,

    echo "CREATE EXTENSION pg_stat_statements;" | psql -U postgres template1

  3. Enable query statistics collection by editing $PGDATA/postgresql.conf where PGDATA is where the database is, e.g.,
    
    # Loading the extension
    shared_preload_libraries = 'pg_stat_statements'
    
    # Increase the max size of the query strings that the database system records
    track_activity_query_size = 2048
    
    # Track all statements including those generated by stored procedures
    pg_stat_statements.track = all
    
    # Log queries that run more than 1000 milliseconds. 
    log_min_duration_statement = 1000
    
    
  4. Restart the database server, i.e.,
    
    sudo systemctl restart postgresql
    



Thursday, October 17, 2019

Increasing PostgreSQL Maintenance Task Speed

I observed maintenance tasks, such as, indexing a large table on a PostgreSQL database took long time. Examining the PostgreSQL configuration, I saw the maintenance_work_mem is 64MB, the default value, e.g.,

postgres=# show maintenance_work_mem;
 maintenance_work_mem
----------------------
 64MB
(1 row)

postgres=#

To speed up the indexing processing, I temporarily increase it to 1GB without restarting the database server. The following transcripts shows the steps:

postgres=# show maintenance_work_mem;
 maintenance_work_mem
----------------------
 64MB
(1 row)

postgres=# ALTER SYSTEM SET maintenance_work_mem='1GB';
ALTER SYSTEM
postgres=# show maintenance_work_mem;
 maintenance_work_mem
----------------------
 64MB
(1 row)

postgres=# SELECT pg_reload_conf();
 pg_reload_conf
----------------
 t
(1 row)

postgres=# show maintenance_work_mem;
 maintenance_work_mem
----------------------
 1GB
(1 row)

postgres=#