Performance Counter Event Errors

Performance Counter Event Errors

https://jawchemist.blogspot.com/

JAWChemist@gmail.com                                                                                   08/07/2019
Ver       20190807-01
Update 20190822 - Additional events and results of using Lodctr /R

Disclaimer

Problem:
Performance counter can cause event errors. The information on fixing them is fragmented. Suggested solutions do not always work and the documentation for the commands used to fix the errors has limited discussion.

Solution:

A parameterized generic resolution procedure to fix performance counter and events [XXXX is the service-name, YYYY is the .ini filename for the service-name, the XXXX folder MUST contain 2 files (symbol file usually .h extension and YYYY.ini]:

Start CMD as administrator (Note: title bar is “Administrator: Cmd”)

CD “C:\windows\INF\XXXX”

Lodctr YYYY.ini

Events

1008 – Perflib - The system cannot find the file specified.

1023 – Perflib -Windows cannot load the extensible counter DLL

2001 – usbperf (a specific service-name) - Unable to read the "First Counter"

8317 – MSSQL$MSSMLBIZ - Cannot query value 'First Counter'

8317 occurred for 2 years after installing Outlook for Business

1008, 1023 and 2001 occurred after Windows update 1903 on 7/10/2019

Update 20190822
1020 - Perlib - The required buffer size is greater than the buffer size passed to the Collect function.                             Service disabled
2017 - PerfOS -Unable to collect NUMA physical memory utilization data.  Only 1 occurance.

Discussion:
Resolution procedure discussion

Some information and caveats should be understood before doing the resolution procedure or it can fail. The most important is that the directory MUST have the 2 files needed in the same directory (see https://docs.microsoft.com/en-us/windows/win32/perfctrs/adding-counter-names-and-descriptions-to-the-registry under key definitions SymbolFile). They are a symbol file (usually .h extension) and an .ini file. If both file are not there it will not be successful. Lodctr is very poor in reporting success or failure of some commands and it may not give an alert but simply an empty line whether it did something or not. The example below uses the specific service-name of usbhub, which has the files usbperf.ini and usbpersym.h. Usbhub and usbperf.ini have to be replaced with the folder and filename for the specific service being fixed.

Specific resolution procedure example

Start CMD

CD “C:\windows\INF\usbhub”

Lodctr usbperf.ini

This was done for each service-name listed by Lodctr /Q and which had a symbol and .ini file available. Although likely not necessary WINMGMT.EXE /RESYNCPERF was run and the computer restarted. The performance monitor showed many more counters (in search box type perfmon.exe, click green plus).

Lodctr and Registry

Lodctr /Q It will give a list of service-names in [] at the beginning of the first line for each service.

The service names will have folders in C:\windows\INF with that name. There can be exceptions where the symbol and ini file are somewhere else. PERLIB seems to contain other information and is not a service-name.

Each service name listed will have 5 or 9 lines. The ones with 5 lines do not have registry entries (First Counter, Last Counter, First Help, Last Help) for that service. These counter were not installed and can give event errors.

The registry HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services will have entries with these service names. It will also have many other non-performance service entries.

When Lodctr INI-filename was run on a service name that had no counter/help entries in the registry, they would appear but ONLY AFTER REFRESHING the registry. Lodctr did not give any indications of a successful registry update.

It will not update the registry if the service-name does not exist in the registry https://docs.microsoft.com/en-us/windows/win32/perfctrs/adding-counter-names-and-descriptions-to-the-registry.

Files .ini and .h (symbol file)

About half of the service name folders in C:\windows\INF DID NOT have both files needed in the same folder. They were usually missing the .ini file and only had the .h file. Subfolders did have ini files. In some cased the size and date were the same. The ini file was copy to the same folder as the .h symbol file before doing the procedure. In some cases, one of the ini files in the subfolders was about an hour newer but the same kilobyte (KB) size. In CMD, the comp command failed because they were not the same size. The .ini files were viewed in Notepad and manually compared. They appeared the same. The most recent was always copied to the folder with the .h file.

The .ini file has an [info] section that lists the drivername=. This is the name used as the service-name listed by Lodctr /Q, the folder name in C:\windows\INF and the registry entry in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services. It also has the symbolfile=, which is the name of the .h file. If the symbol file has another extension like .hxx, it will be seen here.

The symbol file lists symbolic constants for offsets numbers on the right

(https://docs.microsoft.com/en-us/windows/win32/perfctrs/adding-counter-names-and-descriptions-to-the-registry). The numbers start at zero and must be even consecutive numbers (0, 2, 4) for each line. There were 2 symbol files that had duplicate offsets (Rdyboost.ini 2 & 2, Wsearchidxpi 2 & 2, 126 & 126). This may be an issue for these performance counters and cause a problem in events or in the reporting of performance data.

Lodctr /R

As this is widely recommended, it was tried but seemed to be of limited success. Lodctr /? Indicates /R “Rebuilds perf registry from scratch based on current registry settings and backup INI files.” Since the registry was missing entries like “First Counter” for services and the c:\windows\system32\PerfStringBackup.INI only had the PerflIb and PERF_MSSQL$MSSMLBIZ sections, it was not clear how restoring them all is done. https://support.microsoft.com/hr-hr/help/2554336/how-to-manually-rebuild-performance-counters-for-windows-server-2008-6 indicates that all counters may not be recovered and references PerfStringBackup.INI as having the proper information. Mine was missing lots of information. Mine is 32 bit system. Some sites indicate that to use Lodctr /R you should do it in c:\windows\system32 and in c:\windows\sysWOW64 if you have a 64 bit system. Since Lodctr seems to need to be run in the folder that has both the .h and .ini file, for it to work it may need to be run in these folder and have PerfStringBackup.INI with complete information. Lodctr may not give a failure alert if run in some other folder.

Update 20190822

For the services listed by Lodctr /Q and after setting up all the performance counters found by the method describe in “Specific resolution procedure example”  above, the event viewer listed, over time, a couple of other issues that seemed associated with the performance counters (LSM and PerfOS).  They are immediately below.  Running Lodctr /R twice (as reported elsewhere) resulted in some of the performance counters being lost.  The first time gave "Error: Unable to rebuild performance counter setting from system backup store, error code is 2" and the second time gave "Info: Successfully rebuilt performance counter setting from system backup store".  PerfStringBackup.INI had a list of all the counters before running Lodctr /R.  The conditions for Lodctr /R were cmd as Administrator, a capital R and nothing else running. 

The LSM service using perfts.dll terminal service continued to give buffer size errors so this was disabled.  This had happen before rebuilding the counters as describe on this page.  Finding and installing the counters did not correct this.  This seems unrelated to the counters and seems to be some issue in the dll.

The PerfOS service had no .ini or .h files, so it never had any counters installed and the one time error ignored.

Lodctr /R:filename

This appears to restore counters from a previously stored filename (lodctr /S:filename). The file structure of a saved file is the same as https://social.technet.microsoft.com/Forums/ie/en-US/9b01e1a6-d872-4f28-9280-f35d6ca02a9f/lodctr-r-error-code-2?forum=w7itprogeneral I assume this is a more general version of Lodctr /R, which seems to use PerfStringBackup.INI.

MSSQL$MSSMLBIZ as the key to resolution

Event 8317 was an issue for a couple of years after answering yes to installing Outlook for Business and it could not be corrected by uninstalling.

Once the need for both the .ini and .h file to be in the same folder was discovered

1. perf-MSSMLBIZsqlctr.ini had to be found. It was not in the c:\windows\INF\MSSQL$MSSMLBIZ folder. It was in the C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn folder. It was checked to see that it had the same symbolfile= name as the .h file in c:\windows\INF\MSSQL$MSSMLBIZ and copied to that folder.
2. In CMD in the directory c:\windows\INF\MSSQL$MSSMLBIZ, Lodctr perf-MSSMLBIZsqlctr.ini was run. This resolved event 8317. It was then that PERF_MSSQL$MSSMLBIZ section appeared in PerfStringBackup.INI leading me to conclude the procedure would have to be done for each service as they had no entries in this ini file. After running Lodctr, they then appeared in PerfStringBackup.INI.

This indicates that files can go missing, may not be in the correct INF folders and that the backup for Lodctr /R may be incomplete. About half of the service-name folder in INF had only the .h file. I assume that if the .h file and .ini file are in the same directory but not in service-name INF folder, lodctr can still be used.

Find and Findstr

Some sites suggest Findstr or Find to get ini files in the INF subdirectories that have drivername in them.

Findstr only found 3 service name ini files for me even with the /S switch because some of the ini files are utf-16 encode so they have a 00 byte between each character. Findstr only works for ASCII, UTF-8 or ANSI encoded files. Findstr is recommended in https://support.microsoft.com/en-ca/help/300956/how-to-manually-rebuild-performance-counter-library-values by Microsoft.

Find can look in utf-16 encoded ones but unlike Findstr it cannot do subfolders. File explorer search is recommended (drivername name:*.ini). If you use Notepad to view the .h and .ini files, it can mislead you. At the bottom of notepad when Saving As, there is an encoding box and you can see how the file is encoded.

SOLVED:– Mostly

I have 39 service names (ignoring PERFLIB, which does not seem to be a service name). 8 of them only have 5 lines (Lodctr /Q) and do show the First, Last of Counter and Help. These 8 do not have a folder in INF but do have entries in the registry. The registry entries do not have the counters/help keys. There have not been events related to these missing ones. An attempt to find them located elsewhere on my machine did not yield any ini files that seem related to the service-names that had no counter information.

The service-name with missing counters/help: Lsa, MSSCNTRS, PerfDisk, PerfNet, PerfOS, PerfProc, Spooler and Tcpip.