Upgrading TrueNAS/FreeNAS Plugin Jail to Match System Release

When I update a TrueNAS/FreeNAS plugin jail, I encounter the following error,

Updating Plugin

Error: Major ABI change detected, please run 'upgrade' instead. 

The screen shot is below

After some research on the truenas forum, I have gotten a sense what's going on. 

1. First, let's be sure what TrueNAS/FreeNAS release that we are on by issuing the uname -r command, e.g.,
   

   
   TrueNAS $ uanme -r
   12.2-RELEASE-p10
   TrueNAS $
       

2. Second, let's list the plugins. For this, we need to open a TrueNAS/FreeNAS shell, and issue iocage list, e.g.,
   

   
   TrueNAS $ iocage list
   +------+-----------------+-------+--------------+--------------+
   | JID  |      NAME       | STATE |   RELEASE    |    IP4       |
   +======+=================+=======+==============+==============+
   | 4    | calibre         | up    | 11.2-RELEASE | DHCP         |
   +------+-----------------+-------+--------------+--------------+
   | 5    | nextcloud       | up    | 12.2-RELEASE | 198.51.100.1 |
   +------+-----------------+-------+--------------+--------------+
   | 7    | plexmediaserver | up    | 12.2-RELEASE | DHCP         |
   +------+-----------------+-------+--------------+--------------+
   
   TrueNAS $
     

   where we can see that the jail of plugin calibrehas a different system release number.
3. Then, upgrade the jail's release using the iocage upgrade command, e.g.,
   

   
   TrueNAS $ iocage upgrade calibre -r 12.2-RELEASE
       ...
   TrueNAS $
   

Mapping NFS User ID between Linux and FreeNAS/TrueNAS/FreeBSD

I want to mount a NSF share exported on a FreeNAS/TrueNAS/FreeBSD system  on a Fedora Linux system. These two systems have their own independent user  IDs and domains. To display the user name of the NSF share as a user name on the client, I figure that I need to use nfsidmap. For this, I need to solve the following problems.

1. Map the Linux client's NSF domain to that of the FreeNAS/TrueNAS/FreeBSD server. For this, I need to find out the domain name of the FreeNAS/TrueNAS/FreeBSD server. On the server, open a console/terminal, issue the following command

   hostname -d        

   Assume the output is local. We can check whether this matches the NSF domain on the Linux client side, by running the following command on the client side,

   sudo nfsidmap -d 

2. Map the Linux client's user to that of the FreeNAS/TrueNAS/FreeBSD server. For this, I update /etc/idmapd.conf

Finally, I update the Linux client's /etc/idmapd.conf configuration file. Below is an example.

[General]
Domain = local # use the output of "hostname -d" on the server side

[Mapping]
Nobody-User = nobody
Nobody-Group = nobody
        
[Translation]

[Static]
server_user1@local=client_user1

Clearing IPMI Log Entries

I saw this error message on my FreeNAS system,

IPMI SEL low space left: 0 bytes (100% used).

This can happen to any system. In my case, it is that motherboard battery somehow has low voltage and needs replacement. However, one issue is to clear the IPMI SEL log entries to make some space. The following command allows me to clear the log entries,

sudo ipmitool sel clear

Changing IPMI User Password

I was wondering how I would reset or change IPMI user password. The solution is actually simple, i.e., to use ipmitool.

Here is an example. I have a FreeNAS system running a server with a ASROCK Rack IPMI, and would do the following to change a IPMI user's password,

1. Query IPMI users ID
   

   
   sudo ipmitool  user list 1
   

   
   where "1" is the channel number. The outputs resembles the following,
   

   
   
   ID  Name             Callin  Link Auth  IPMI Msg   Channel Priv Limit
   1                    false   false      true       ADMINISTRATOR
   2   admin            false   false      true       ADMINISTRATOR
   3                    true    false      false      NO ACCESS
   4                    true    false      false      NO ACCESS
   5                    true    false      false      NO ACCESS
   6                    true    false      false      NO ACCESS
   7                    true    false      false      NO ACCESS
   8                    true    false      false      NO ACCESS
   9                    true    false      false      NO ACCESS
   10                   true    false      false      NO ACCESS
   

   
   where the admin user's ID is 2.
   
   
2. Change IPMI user password. The following is to change the "admin" user's password to admin123.
   

   
   sudo ipmitool  user set password 2 admin123
   

   
   where "2" is the ID of the admin user. 
   

We can also do the above on a Linux system. Note that these are for the host's IPMI.

Finding Disk's Serial Numbers on FreeNAS (or FreeBSD)

To list hard disk drives' serial numbers in a FreeNAS or a FreeBSD system, we can run the following command

geom disk list | grep ident

Extending a Linux Virtuam Machine File System on a FreeNAS zvol.

I have FreeNAS system where I run a Fedora Linux system on a Virtual Machine. I want to increase the size of a file system on the Linux VM. The file system is on a FreeNAS zvol. Assuming that the file system is mounted at /mnt, and the file system is a Linux Extended File System. The steps are as follows to increase the size of the partition.

1. Increase the zvol via FreeNAS user interface
2. Resize the partition at the Linux VM. Go to Linux VM, run parted. See transcript below
   

   
      $ parted /dev/sdc
      (parted) resizepart 1 100%
      (parted) quit
      $
   

   
   
3. Resize the file system on the partition we just resize. Below are the steps
   

   
      $ e2fsck -f /dev/sdc1
      $ resize2fs /dev/sdc1
   

Updating Plex Media Server to Latest FreeNAS Plugin on Command Line in Jail

There are two methods to upgrade Plex Media Server in FreeNAS Jail. 

1. Method 1. Upgrading Plex Media Server to Latest FreeBSD Release. Using this method, you don't have to wait for FreeNAS developers to upgrade Plex Media Server in the plugin repository.
2. Method 2. Upgrading Plex Media Server to Latest FreeNAS Plugin. 

This post is about Method 2, and documents the steps to upgrade Plex Media Server on Command Line.

1. Login or open FreeNAS shell
2. List jails
   

   
   $ jls
   

   
   The output looks like the following
   

   
   $ jls
      JID  IP Address  Hostname     Path
        1              plex         /mnt/Storage/iocage/jails/plex/root
   $

3. Open a shell inside the jail. The command is as follows,
   

   
   jexec ${JID} csh
   

   
   In the above, the jail ID (JID) is 1. Since we can only the csh as root, we open a shell inside the jail as follows

   
   $ sudo jexec 1 csh
   

4. Upgrading the plexmediaserver package. Inside the jail, we do
   

   
   # pkg upgrade plexmediaserver
   

   
   If the installed version is already latest, we should observe something similar to the following,
   

   
   root@plex:/ # pkg upgrade plexmediaserver
   Updating iocage-plugins repository catalogue...
   iocage-plugins repository is up to date.
   All repositories are up to date.
   Checking integrity... done (0 conflicting)
   Your packages are up to date.
   root@plex:/ #
   

   
   However, if there is a newer version, the upgrade will commerce.
   
   
5. Restart Plex Media Server if it gets upgraded as follows inside the jail,
   

   
   # service plexmediaserver restart
   

Replacing Corrupt Hard Drive for FreeNAS

An ATA hard drive can fail without being detected by the SMART system in time. At a FreeNAS 11 system I observe when this happens, the drive disappears from the zpool disk list while FreeNAS reports an alert typically like the following,

The volume MyZpool state is DEGRADED: One or more devices has been taken offline 
by the administrator. Sufficient replicas exist for the pool to continue 
functioning in a degraded state.

Since the disk drive disappears from the zpool disk list, the method to replace the disk drive in the FreeNAS guide won't work -- after you replaced the corrupted hard disk drive, if you bring the disk drive online, the disk will be "UNAVAIL"; and if you attempt to replace it via the FreeNAS's Web interface, there will be no disk for you to choose from. If you run "zpool status", you will observe a long sequence of digits instead of the partition as the following example:

$ zpool status
  pool: MyZpool
 state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
        the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
   see: http://illumos.org/msg/ZFS-8000-2Q

config:

        NAME                                            STATE     READ WRITE CKSUM
        MyZpool                                         DEGRADED     0     0     0
          raidz2-0                                      DEGRADED     0     0     0
            gptid/11111111-2222-3333-4444-555555555555  ONLINE       0     0     0
            gptid/22222222-2222-3333-4444-555555555555  ONLINE       0     0     0
            99999999999999999999                        UNAVAIL      0     0     0  was /dev/ada2
            gptid/33333333-2222-3333-4444-555555555555  ONLINE       0     0     0
            gptid/44444444-2222-3333-4444-555555555555  ONLINE       0     0     0
            gptid/55555555-2222-3333-4444-555555555555  ONLINE       0     0     0

errors: No known data errors

  pool: freenas-boot
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:10:03 with 0 errors on Tue Jan 15 03:55:03 2019
config:

        NAME        STATE     READ WRITE CKSUM
        freenas-boot  ONLINE       0     0     0
          da0p2     ONLINE       0     0     0

errors: No known data errors

The method to resolve this is via command line. Below is the steps starting from replacing the physical hard drive.

1. Replace the physical hard drive. For this, shutdown the machine if necessary. 
2. Partition the hard drive. Following is an example where we assume the disk is /dev/ada2.

   
   # create gpt called ada2
   sudo gpart create -s gpt ada2
   # create a 2G swap partition (it will be ada2p1)
   sudo gpart add -i 1 -b 128 -t freebsd-swap -s 2G ada2
   # create a second partition using the rest of the space (it will be ada2p2)
   sudo gpart add -i 2 -t freebsd-zfs ada2
   # replace disk labeled "99999999999999999999" by ada2p2. See the error message example above. 
   sudo zpool replace MyZpool 99999999999999999999 ada2p2
   

Once the above is completed, FreeNAS will immediately start resilvering the zpool.

FreeNas Plugins: "No Entry Has Been Found"?

A FreeNas system (FreeNAS-11.1-U6) reports "No entry has been found" on its Web interface. However, running pbi_browser seems to yield the list of plugins, for instance,

$ sudo /usr/local/sbin/pbi_browser  --viewall

Using RepoID: 001
Listing all available PBIs
------------------------------------
Application: CouchPotato
Version: autoupdate.9_5
Category: Network - P2P
Created: 20160212 060001
RootInstall: NO
Arch: amd64
Author: RuudBurger
URL: https://couchpota.to/
License: GPLv3
Type: Graphical
Keywords: net,p2p,share,nzb
Icon: /var/db/pbi/repo-icons/a5a51ab301916fcacfa9029fc96adff2-CouchPotato.png
Description: An automatic NZB and torrent downloader

To install this PBI:
# pbi_add --rArch amd64 --repo 001 -r CouchPotato

------------------------------------
Application: Emby
Version: 3.3.1.0
 

The solution that worked for me is simply to reboot the system.

Finding a Disk's Serial Number in FreeNAS

When a harddisk pool in FreeNAS encounters many errors, FreeNAS would report errors. In this case, we need to figure out which physical disk may have problem. The following steps help us find out serial numbers of the disk,, in conjunction with the BIOS information, we may figure which physical disk has problem.

# show a list of disks with their GPT IDs
zpool status
# show the mapping beetween GPT IDs and device names
glabel status
# from above two steps, we can know which physical device may have 
# problem, e.g., /dev/ada5p2
sudo  smartctl -a /dev/ada5 | grep "Serial Number"