Thursday, April 30, 2015

Node Package Manager (npm) Complaining "Unknown Error" from Linux Guest on Windows Host

I ran a npm install on a Linux guest running inside a Windows host and received an "Unknown Error". The error message looks like the following,


npm http 200 https://registry.npmjs.org/mime-db/-/mime-db-1.8.0.tgz
npm ERR! Error: ENOENT, lstat '/foo/node_modules/sauce-tunnel/vendor/linux/bin/sc'
......
npm ERR! node -v v0.10.25
npm ERR! npm -v 1.3.10
......
npm ERR! fstream_type File
npm ERR! fstream_class FileWriter
npm ERR! code ENOENT
npm ERR! errno 34
npm ERR! fstream_stack /usr/lib/nodejs/fstream/lib/writer.js:284:26
npm ERR! fstream_stack Object.oncomplete (fs.js:107:15)
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /foo/node_modules/something/npm-debug.log
npm ERR! not ok code 0


The solution is to run the following,

npm cache clean
npm install --no-bin-links
The cause of these type of errors, apparently is that when we install a node package to a shared Windows directory on a Linux guest and the underlining Windows filesystem does not know how to handle the Linux/Unix links, according to the post here.

Friday, April 24, 2015

Restoring Corrupted Windows Components Detected When Performing SFC, DSIM, and System Update

Using the System File Checker tool (SFC) and the Deployment Image Servicing and Management tool (DISM), we may fix a corrupted Windows system or a corrupted Windows component store. However, there are some cases the tools would fail to fix the problems. In addition, when performing a Windows update, we may also experience some problem. This post documents how we may locate a problem and manually fix the problem. This post is complementary to other two posts.

When we use SFC to repair missing or corrupted system files, SFC may report that it could not fix some corrupted files. Below is an example,

  C:\WINDOWS\system32>sfc /scannow

  Beginning system scan.  This process will take some time.

  Beginning verification phase of system scan.
  Verification 100% complete.

  Windows Resource Protection found corrupt files but was unable to fix some
  of them. Details are included in the CBS.Log windir\Logs\CBS\CBS.log. For
  example C:\Windows\Logs\CBS\CBS.log. Note that logging is currently not
  supported in offline servicing scenarios.

  C:\WINDOWS\system32>
In the above, the output of the SFC command contains,

  Windows Resource Protection found corrupt files but was unable to fix 
  some of them.
which indicates obviously that some files are corrupted and unfortunately SFC cannot fix them for you. The first and an easy attempt to fix the issue, in my opinion, would be to run a DISM command, for instance,

  C:\WINDOWS\system32>dism.exe /Online /Cleanup-Image /RestoreHealth

  Deployment Image Servicing and Management tool
  Version: 6.3.9600.17031

  Image Version: 6.3.9600.17031

  [==========================100.0%==========================]
  The restore operation completed successfully. The component store corruption was
  repaired.
  The operation completed successfully.

  C:\WINDOWS\system32>


Then, run SFC again. With good fortune, SFC may report that it can now fix the corrupted files, as demonstrated below,

  C:\WINDOWS\system32>sfc /scannow

  Beginning system scan.  This process will take some time.

  Beginning verification phase of system scan.
  Verification 100% complete.

  Windows Resource Protection found corrupt files and successfully repaired
  them. Details are included in the CBS.Log windir\Logs\CBS\CBS.log. For
  example C:\Windows\Logs\CBS\CBS.log. Note that logging is currently not
  supported in offline servicing scenarios.

  C:\WINDOWS\system32>


However, when we use DISM to recover from a corrupted Windows component store, DISM may also report that it could not fix some corrupted component store.

Even if the above two yield no error, a Windows Update may still fail. For instance, when I attempt to apply the KB3000850 update to my Windows 8.1 system, I encountered an error -- the update appears to be applied correctly; however, when the system reboots, it complains that the update failed to be applied successfully and the system rolled back and undid the change.

There are three logs that we typically examine for clues to identify corrupted component or component store.
The procedure that we can use to locate and fix the problem is outlined in this post.

The procedure is a manual one. In this post, let us examine two example problems and the methods to fix them. The first example is to examine CBS.log to obtain clues and fixing strategies. The second example is to examine setupapi.dev.log for clues to fix the problem.

  1. Example 1. After the DISM command fails to the /RestoreHealth operation, we examine %WinDir%\Logs\CBS\CBS.log. By searching "Total Detected Corruption" in the log file, we would find the following logging message,
    
    Checking System Update Readiness.
    
    (p) CSI Payload Corrupt   amd64_microsoft-windows-lddmcore_31bf3856ad364e35_6.3.9600.16438_none_9cc10b53b757a2be\dxgkrnl.sys
    Repair failed: Missing replacement payload.
    (p) CSI Payload Corrupt   amd64_microsoft-windows-lddmcore_31bf3856ad364e35_6.3.9600.16408_none_9ce17b17b73f4eeb\dxgkrnl.sys
    Repair failed: Missing replacement payload.
    
    Summary:
    Operation: Detect and Repair 
    Operation result: 0x0
    Last Successful Step: Entire operation completes.
    Total Detected Corruption: 2
       


    The message indicates that two files are corrupted. Both of the files happen to be two versions of dxgkrnl.sys. In this example, we demonstrate that we obtain a copy of the file from a Windows update manually. The CBS.log cites many Windows updates. It is infeasible to download all of them and see if the update contains a copy of the file. The method I used to locate the appropriate Windows update is to search the Windows version number. In this case, according to the message, the Windows version number is 6.3.9600.16438 in the first CSI Payload Corrupt message. We search "6.3.9600.16438 dxgkrnl.sys" using Google, as below,
    
       https://www.google.com/search?q=6.3.9600.16438+dxgkrnl.sys
       
    From the search result, it appears that Windows Update KB2887595. We then locate the stand-alone update package of the Windows Update from the Windows Download Center. The search string I used to locate the Windows Update is "KB2887595 x64" since my Windows system is a 64-bit version of Windows 8.1. Having downloaded the stand-alone update package of the Windows Update, we will obtain the copy of the version of dxgkrnl.sys and replace the corrupted copy with the downloaded one using the steps as follows, assuming that the Window Update Windows8.1-KB2887595-v2-x64.msu is in directory C:\Temp,
    
    C:\Windows\System32>cd C:\Temp
    
    C:\Temp>mkdir KB2887595
    
    C:\Temp>expand -f:* Windows8.1-KB2887595-v2-x64.msu KB2887595 > nul
    
    C:\Temp>cd KB2887595
    
    C:\Temp\KB2887595>mkdir KB2887595CAB
    
    C:\Temp\KB2887595>expand -f:* Windows8.1-KB2887595-v2-x64.cab KB2887595CAB > nul
    
    
    C:\Temp\KB2887595>dir KB2887595CAB\amd64_microsoft-windows-lddmcore_31bf3856ad364e35_6.3.9600.16438_none_9cc10b53b757a2be
     Volume in drive C is Windows8_OS
     Volume Serial Number is 282A-6B47
    
     Directory of C:\Temp\KB2887595\KB2887595CAB\amd64_microsoft-windows-lddmcore_31bf3856ad364e35_6.3.9600.16438_none_9cc10b53b757a2be
    
    04/25/2015  04:39 PM    <DIR>          .
    04/25/2015  04:39 PM    <DIR>          ..
    08/22/2013  08:36 AM           212,992 cdd.dll
    10/19/2013  05:13 AM         1,530,200 dxgkrnl.sys
    10/03/2013  10:07 AM           382,808 dxgmms1.sys
    08/22/2013  02:44 AM             2,465 lddmcore.ptxml
                   4 File(s)      2,128,465 bytes
                   2 Dir(s)  127,155,011,584 bytes free
    
    C:\Temp\KB2887595>
       


    Clearly, we have successfully obtained a good copy of the corrupted file. Now we simply to copy the file to its component store under %WinDir%\WinSxS as demonstrated below,
    
    C:\Temp>takeown /f "%windir%\winsxs" /a
    C:\Temp>icacls "%windir%\winsxs" /grant:r *S-1-5-32-544:(OI)(CI)(F) /q
    C:\Temp\KB2887595>copy KB2887595CAB\amd64_microsoft-windows-lddmcore_31bf3856ad364e35_6.3.9600.16438_none_9cc10b53b757a2be\dxgkrnl.sys %WinDir%\WinSxS\amd64_microsoft-windows-lddmcore_31bf3856ad364e35_6.3.9600.16438_none_9cc10b53b757a2be
    Overwrite C:\WINDOWS\WinSxS\amd64_microsoft-windows-lddmcore_31bf3856ad364e35_6.3.9600.16438_none_9cc10b53b757a2be\dxgkrnl.sys? (Yes/No/All): a
    
            1 file(s) copied.
    
    C:\Temp\KB2887595>
       


    Similarly, we can conduct a web search for the second corrupted file via Google as follows,
    
       https://www.google.com/search?q=6.3.9600.16408+dxgkrnl.sys
       

    From the search we learn that we could locate the file in the Windows Update KB2883200. Following the same procedure, we can replace the corrupted file with a good copy.
  2. Example 2. The second example is to resolve the update failure problem when I tried to apply the KB300850 update. Examining %WinDir%\Inf\setupapi.dev.log, I found the following messages,
    
         sto: {Publish Driver Package: C:\WINDOWS\System32\DriverStore\FileRepository\wiaek002.inf_amd64_5b5a15ef9a58384c\wiaek002.inf} 07:54:15.316
         sto:      Driver Package = wiaek002.inf_amd64_5b5a15ef9a58384c
         idb:      {Publish Driver Package: C:\WINDOWS\System32\DriverStore\FileRepository\wiaek002.inf_amd64_5b5a15ef9a58384c\wiaek002.inf} 07:54:15.316
         idb:           Changing active driver package from 'wiaek002.inf_amd64_57f9361b96ceea4b' to 'wiaek002.inf_amd64_5b5a15ef9a58384c'.
         cpy:           Unpublished 'wiaek002.inf'.
    !    inf:           Unable to load INF: 'C:\WINDOWS\System32\DriverStore\FileRepository\wiaek002.inf_amd64_57f9361b96ceea4b\wiaek002.inf'(00000003)
    !    inf:           Error 3: The system cannot find the path specified.
    !!!  inf:           Failed to open INF file "C:\WINDOWS\System32\DriverStore\FileRepository\wiaek002.inf_amd64_57f9361b96ceea4b\wiaek002.inf". Error = 0x00000003, Line = 0
    !!!  sto:           Failed to get driver package file list. Error = 0x00000003, Filename = C:\WINDOWS\System32\DriverStore\FileRepository\wiaek002.inf_amd64_57f9361b96ceea4b\wiaek002.inf
    !!!  idb:           Failed to update driver package files for 'wiaek002.inf_amd64_57f9361b96ceea4b'. Error = 0x00000003
    !!!  idb:           Failed to publish 'C:\WINDOWS\System32\DriverStore\FileRepository\wiaek002.inf_amd64_5b5a15ef9a58384c\wiaek002.inf'. Error = 0x00000003
         idb:      {Publish Driver Package: exit(0x00000003)} 07:54:15.316
    !!!  sto:      Failed to publish driver package. Error = 0x00000003
         sto: {Publish Driver Package: exit(0x00000003)} 07:54:15.316  
      
    From the messages, we can learn that the wiaek002.inf file failed to be published in directory C:\WINDOWS\System32\DriverStore\FileRepository\wiaek002.inf_amd64_5b5a15ef9a58384c\wiaek002.inf

    Following the observation, we manually "publish" the driver.
    
          mkdir C:\WINDOWS\System32\DriverStore\FileRepository\wiaek002.inf_amd64_5b5a15ef9a58384c
          copy C:\WINDOWS\Inf\wiaek002.inf C:\WINDOWS\System32\DriverStore\FileRepository\wiaek002.inf_amd64_5b5a15ef9a58384c\wiaek002.inf
        

    Then we reapply the KB300850 update again. In my case, it is a success.

Fixing Windows Update Error for KB3000850

Microsoft issued KB3000850, a 700MB patch. Microsoft labeled it as an optional update in November 2014 and has elevated it to . Microsoft has elevated it to an important update. This update creates problems for many, myself include because the update failed to install on one of my Windows 8.1 system.

Examining the search results from Google, it appears that the installation failures were generally the results of two.
  • Some components of Windows are corrupted
  • Some components of 3rd party software, e.g, hardware drivers are either corrupted or with bugs in installation procedures

With some efforts, I managed to install the update successfully on the Windows systems with problem. The procedure is as follows,
  • Use the System File Checker tool (SFC) to repair missing or corrupted system files. Typically, we follow the procedure below.
    1. Open a Windows Command Prompt with elevated permission, i.e., run the Windows Command Prompt as administrator.
    2. Issue the following command using the System File Checker tool SFC.
      
                  sfc /scannow
              
      The above command scans integrity of all protected system files and repairs files with problems when possible.
  • Use the Deployment Image Servicing and Management (DISM) tool to clear up Windows system image and recover the corrupted system image. Typically, we follow the procedure below.
    1. Open a Windows Command Prompt with elevated permission, i.e., run the Windows Command Prompt as administrator.
    2. Issue the following command using the Deployment Image Servicing and Management (DISM) tool.
      
                  Dism.exe /Online /Cleanup-Image /RestoreHealth
              
      In the above command, the /Online option indicates that the command is working on the currently running Windows, i.e., the system is currently "Online". Nevertheless, the above command scans the current running Windows for component store corruption, and then perform repair operations automatically. See the post for more examples using dism
  • Fix issues with 3rd party drivers. This step is typically a manual operation. In my case, I have a Kodak printer and installed the driver to the Kodak printer. It appears that the Kodak driver installation procedure has some issues and prevented the KB3000850 update from being installed successfully.
    In my case, the steps I took to fix the problem were as follows,
    
      mkdir %WinDir%\System32\DriverStore\FileRepository\wiaek002.inf_amd64_5b5a15ef9a58384c
      copy %WinDir%\Inf\wiaek002.inf %WinDir%\System32\DriverStore\FileRepository\wiaek002.inf_amd64_5b5a15ef9a58384c
          

    The clue of this solution comes from the log file %WinDir%\Inf\setupapi.dev.log and two references that helped a great deal are these two posts. However, the steps would be different from case to case.
    A more detailed discussion is in another post.

Wednesday, April 22, 2015

Checking and Restoring Windows Images

Deployment Image Servicing and Management tool (DSIM) is a command-line tool used to install, uninstall, configure, and update Windows features, packages, drivers, and international settings primarily on Windows images offline before deployment. Some DISM servicing commands are also available for running operating systems. The basic syntax for dism is (in a single line),

DISM.exe {/Image:<path_to_image> | /Online} [dism_options] 
                         {servicing_command} [<servicing_argument>]

That is to day, for servicing a Windows image offline, the syntax is,

DISM.exe /Image:<path_to_image> {servicing_command} [<servicing_argument>]
and for servicing a running Windows operating system, the syntax is,

DISM.exe /Online [dism_options] {servicing_command} [<servicing_argument>]
For instance, to service a running Windows, we can use /Cleanup-Image servicing command to perform cleanup and recovery operations on the image, which are commonly called for to deal with corrupted windows system.
  • Use servicing argument /CheckHealth to check whether the image has been flagged as corrupted by a failed process and whether the corruption can be repaired.
  • Use servicing argument /ScanHealth to scan the image for component store corruption.
  • Use servicing argument /RestoreHealth to scan the image for component store corruption, and then perform repair operations automatically.
More concretely, we can run the following commands from Windows Command Prompt with elevated permission,

Dism.exe /Online /Cleanup-Image /ScanHealth

Dism.exe /Online /Cleanup-Image /CheckHealth

Dism.exe /Online /Cleanup-Image /RestoreHealth

Alternatively, from Windows Powershell, run,

Repair-WindowsImage -Online -CheckHealth

Repair-WindowsImage -Online -ScanHealth

Repair-WindowsImage -Online -RestoreHealth

Tuesday, April 21, 2015

Google Mobile Friendly Seaches

Google made the following announcement,
Starting April 21, we will be expanding our use of mobile-friendliness as a ranking signal. This change will affect mobile searches in all languages worldwide and will have a significant impact in our search results. Consequently, users will find it easier to get relevant, high quality search results that are optimized for their devices.

For future web development and existing web site optimization, we want to know if a site is "mobile-friendly", for which it is perhaps a good practice to check out Google's blog post entitled Finding more mobile-friendly search results.
Google provides two tools,
These tools are excellent. However, it does not appear that they could be used before deployment. Perhaps, a good starting point is to examine the Google's Mobile-Friendly Web Development Guide.

Monday, April 13, 2015

Cost of Higher Education: New York Times and Slate Editions

Recently, New York Times published an op-ed opinion piece entitled "The Real Reason College Tuition Costs So Much" (the piece). The piece assigns a single factor to the cause of the rising college tuition, that is, the bloating university and college administration. The piece is interesting to me. I would like to drop a note on the piece.

The piece definitely won many praises -- who does not hate rising college tuition cost? I would like to say that the piece is an excellent piece simple because the piece certainly garnered many serious discussions. A few readers wrote to the New York Times expressed their opinions toward the piece. Among the opinions, some counters the argument made in the piece by arguing some university administration has become more efficient, and some argues that the bloating administrative cost is partially a result of the complexity of the higher education business, such as, the increased complexity of regulatory compliance. The Inside Higher ED has a blog post on this piece as well. The blog post refutes the center thesis expressed in the piece.

The discussions on the piece that I enjoy reading and listening the most are an article in Slate.com and a podcast from Slate.com. The article and the podcast point to a few important questions.

  1. Is the cost of college education really rising? If it is rising, how much rising is it?
  2. What are the factors driving the rising college education cost?

To answer the first question, we have to come out some metrics. There are at least two metrics.

  • Total higher education expenditure (or cost), including tuition & fees paid by students and government expenditure and subsidy
  • Higher education expenditure per Full-Time-Equivalent student (FTE)

It appears that we can have the following observations.

  • The Slate.com article and podcast suggest that the total higher education expenditure indeed increases over time. However, so does the number of students enrolled. Then, although the higher education expenditure per FTE is rising, it does not rise much.
  • Indeed, the university & college administrations have gotten bloated. It is a contributing factor of the rising cost of higher education. However, the bloat is partially a result that universities and colleges must deal with increased complexity of regulatory compliance.

One may ask the question, whether we can make higher education more efficient by reducing the cost per FTE.

  • We do not seem to have a solution to reduce the cost amid recent technological advancement in delivering education content and service. Some cites the so-called Baumol Effect to explain why we cannot be more efficient. Nn article in Forbes has a discussion on the Baumol Effect.
  • It is questionable that universities and colleges have any real incentive to reduce the cost per FTE. Some points out the prestige and ranking of a college is positively correlates to the tuition -- the more expensive a college is, the higher rank or reputation the college may enjoy, which is important to recruit more and better students.
  • Some blames the Federal Financial Aid program, such as an Wall Street Journal article and the Forbes article just mentioned above.

Wednesday, April 8, 2015

Configuring SELinux to Allow HTTPD Scripts and Modules to Establish TCP Connections

In a PHP script that establishes database connection to a PostgreSQL database via TCP socket, I encountered a SELinux denial. The message in the error log is,


SELinux is preventing /usr/sbin/httpd from name_connect access on the tcp_socket.


A quick fix is to allow HTTPD scripts and modules to connect to the network using TCP, i.e., run the following,


sudo setsebool -P httpd_can_network_connect 1

By the way, the relevant system information is as follows,

  • Operating System (output of uname -sr): Linux 3.12.9-201.fc19.x86_64
  • SELinux Policy Module: selinux-policy.noarch, version 3.12.1-74.17.fc19