builtin vs. kernel modules

Issues related to applications and software problems and general support
Post Reply
mathog
Posts: 258
Joined: 2008/07/09 23:52:06

builtin vs. kernel modules

Post by mathog » 2020/03/23 22:29:28

If one runs "lspci -k" there are often entries like this:

Code: Select all

02:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5722 Gigabit Ethernet PCI Express
	Subsystem: Dell Device 04de
	Kernel driver in use: tg3
	Kernel modules: tg3
03:03.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200eW WPCM450 (rev 0a)
	Subsystem: Dell Device 04de
	Kernel driver in use: mgag200
	Kernel modules: mgag200
which I have always found ambiguous - is the driver in use builtin or is it actually that named module. So a script was written to do the same thing (bottom of post) and it gave this output:

Code: Select all

./map_hardware_to_drivers.pl PciIDsFile
Checking this machine for drivers
Getting Device ID list from https://pci-ids.ucw.cz/v2.2/pci.ids
Getting PCI ID to builtin mappings from /sys/bus/pci/drivers
Getting PCI ID to kernel module name mappingsChecking pci and device IDs from lspci -n
pci_id  dev_id    driver description
00:00.0 8086:0100 snb_uncore(builtin) "2nd Generation Core Processor Family DRAM Controller"
00:1a.0 8086:1C2D <none> "6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2"
00:1c.0 8086:1C10 <none> "6 Series/C200 Series Chipset Family PCI Express Root Port 1"
00:1c.4 8086:1C18 <none> "6 Series/C200 Series Chipset Family PCI Express Root Port 5"
00:1d.0 8086:1C26 <none> "6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1"
00:1e.0 8086:244E <none> "82801 PCI Bridge"
00:1f.0 8086:1C52 lpc_ich(module) "C202 Chipset LPC Controller"
00:1f.2 8086:1C02 ahci(module) "6 Series/C200 Series Chipset Family 6 port Desktop SATA AHCI Controller"
00:1f.3 8086:1C22 i2c_i801(module) "6 Series/C200 Series Chipset Family SMBus Controller"
02:00.0 14E4:165A tg3(module)/tg3(builtin) "NetXtreme BCM5722 Gigabit Ethernet PCI Express"
03:03.0 102B:0532 mgag200(module)/mgag200(builtin) "MGA G200eW WPCM450"
Done
This shows that for those two devices there are both builtin and module drivers. However it still doesn't quite answer the question of which one is in use. Running lsmod does show that the tg3 and mgag200 drivers are loaded, so presumably those are the ones in use.

But why? Is the tg3 driver which is built in some subset of the module driver? If not, why bother having the module at all if all of its functions are already built into the kernel?

EDIT. Hmm. modules.builtin does NOT contain tg3 and mgag200. So showing up in /sys does not necessarily indicate a builtin driver. Looks like I need to revisit the logic in that script...

Here is the script:

Code: Select all

#!/usr/bin/perl
#Name:    map_hardware_to_drivers.pl
#version: 1.0.0
#Date:    2020-03-12
#Author:  David Mathog
#
#  Map the pci hardware list to drivers for the running kernel
#
#  assumes wget, lspci, and access to the internet are available
#  If access is not available specify one parameter to the already downloaded
#  PCI IDs file.
#
#
use strict;
use warnings;
my $IDLIST="https://pci-ids.ucw.cz/v2.2/pci.ids";
my $real_filename="/tmp/pci.ids";
my $kernel=`uname -r`;
chomp $kernel;
my $MODALIAS="/lib/modules/$kernel/modules.alias";
my %dev_hash_longname;
my %dev_hash_module;
my %pci_hash_builtin;
my $count=0;
my $vendor="";
my $device="";
my $long_name="";
my $module_name="";
my $pci_id;
my $dev_id;
my $ignore;

my $num_args = $#ARGV;
if ($num_args != 0) {
    print 
     "Usage: map_hardware_to_drivers.pl PciIDsFile\n\n",
     "If internet access is available let PciIDsFile = \"-\" and it will be downloaded\n",
     "automatically.  Otherwise on some machine do:\n\n",
     "   wget -O PciIDsFile $IDLIST\n\n",
     "Then copy that file by some means to the target and specify\n",
     "the file name on the command line.\n";
    exit;
}
my $filename=$ARGV[0];



if($filename eq '-'){
   my $messages = `wget -q -O $real_filename $IDLIST 2>/dev/null`;
   if($messages){
      print
        "Some problem running:\n\n",
        "  wget -q -O $real_filename 2>/dev/null\n\n",
	"returned:\n",
	"$messages\n";
      exit;
   }
}
else {
   $real_filename = $filename;
}

#######################
open(FH, $real_filename) or die "could not open file $real_filename";

print "Checking this machine for drivers\n";
print "Getting Device ID list from $IDLIST\n";
# Syntax in this file:
#   #comment line
#   <blank lines>
#   vendor  vendor_name
#	device  device_name				<-- single tab
#		subvendor subdevice  subsystem_name	<-- two tabs
#   Only the vendor and device lines are retained
#(This will obtain all PCI ID -> long name mappings even if the current kernel does
#not have the information.)
while (my $line = <FH>){
    chomp $line;
    if(! $line){ next; }
    if(substr($line,0,1) eq "#"){ next; }
    if(substr($line,0,2) eq "\t\t"){ next; }
    if(substr($line,0,1) eq "\t"){ #device line
       $device = substr($line,1,4);
       $long_name = substr($line,7);
       $dev_id = $vendor . ":" . uc $device;
       $dev_hash_longname{$dev_id} = $long_name;
    }
    else { #vendor line
       $vendor = uc substr($line,0,4);
    }
}
close(FH);
#       my ($dev_type, $dev_id, $dev_driver) = split(/\s+/,$line);

#print "Getting pci IDs and long names from lspci\n";
#foreach my $line (`lspci`){
#    chomp $line;
#    my $starts = index($line," ");
#    my $pci_id = uc substr($line,0,$starts);
#    my $ln = substr($line,$starts+1);
#    $dev_hash_longname{$pci_id}=$ln;
#}

#######################
# get all the PCI ID to builtin mappings.
print "Getting PCI ID to builtin mappings from /sys/bus/pci/drivers\n";
foreach my $line (`find /sys/bus/pci/drivers`){
    chomp $line;
    my $starts = index($line,"/0000:");
    if($starts != -1){
       my $pci_id = uc substr($line,$starts + 6);
       my $delim = rindex(substr($line,0,$starts-1),"/");
       my $driver= substr($line,$delim+1,$starts - $delim -1);
#       print "pci_id >$pci_id< driver >$driver<\n";
       $pci_hash_builtin{$pci_id}=$driver . "(builtin)";
    }
}

#######################
# get all the PCI ID to module name mappings.  Ignore subvendor and subdevice information.
# This uses the "alias pci:" lines which have the syntax:
#   alias pci:vVENDORdDEVICEsvSUBVENDORsdSUBDEVICEscIGNOREiIGNORE module_name
# extract the VENDOR and DEVICE files to construct a PCI ID like 1234:ABCD
# Use that to store the module_name value in a hash.
#

open(FH, $MODALIAS) or die "could not open file $MODALIAS";
print "Getting PCI ID to kernel module name mappings";

my $next_delim;
while (my $line = <FH>){
    chomp $line;
    if(substr($line,0,10) eq 'alias pci:'){
#       print "line: $line\n";
       if(substr($line,11,1) eq "*"){ next; }
       $next_delim=index($line,"d",11);
       if($next_delim == -1){ die "File $MODALIAS has pci: line with unexpected syntax: $line"; }
       $vendor = substr($line,15,4);
       $next_delim=index($line,"sv",20);
       if($next_delim == -1){ die("File $MODALIAS has pci: line with unexpected syntax: $line"); }
       $device = substr($line,24,4);
       $dev_id = $vendor . ":" . $device;
       $next_delim=index($line," ",28);
       if($next_delim == -1){ die "File $MODALIAS has pci: line with unexpected syntax: $line"; }
       $module_name = substr($line,$next_delim + 1);
       if(defined($dev_hash_module{$dev_id})){ next; } #subvendor/subdevice lines
       $dev_hash_module{$dev_id} = $module_name . "(module)";
       $count++;
#       print "  parsed $dev_id $module_name\n";
    }
}
close(FH);

print "Checking pci and device IDs from lspci -n\n";
print "pci_id  dev_id    driver description\n";
foreach my $line (`lspci -n`){
    chomp $line;
    ($pci_id, $ignore, $dev_id)= split /\s/,$line;
    $dev_id = uc $dev_id;
    my $description = (defined($dev_hash_longname{$dev_id}) ? $dev_hash_longname{$dev_id} : "unknown");
    my $module = (defined($dev_hash_module{$dev_id}) ? $dev_hash_module{$dev_id} : "");
    my $builtin = (defined($pci_hash_builtin{$pci_id}) ? $pci_hash_builtin{$pci_id} : "");
    if(!$module){
       if($builtin){
          $module = $builtin;
       }
       else {
          $module = "<none>";
       }
    }
    else {
       if($builtin){
          $module .= "/" . $builtin;
       }
    }
    print "$pci_id $dev_id $module \"$description\"\n";
}
if($filename eq "-"){
   unlink($real_filename);
}
print "Done\n";

mathog
Posts: 258
Joined: 2008/07/09 23:52:06

Re: builtin vs. kernel modules

Post by mathog » 2020/03/24 19:16:01

The revised script (see bottom of post) works better. On the machines tested so far it seems to be correctly identifying kernel vs module drivers. However, if "lspci -k" shows shpchp (hot pluggable) that will not be indicated. I have not yet been able to figure out how lscpi determines shpchp (so far only observed on an IBM x3455 7984). If a device uses more than one driver at a time only one will be shown.

Example (PowerEdge T110 II)

Code: Select all

./map_hardware_to_drivers.pl PciIDsFile
Checking this machine for drivers
Getting Device ID list from https://pci-ids.ucw.cz/v2.2/pci.ids
Getting module list from /sys/bus/pci/drivers
Getting PCI ID to builtin mappings from /sys/bus/pci/drivers
Getting DEV ID to kernel module mappings from /lib/modules/4.18.0-147.5.1.el8_1.x86_64/modules.alias
Summarize pci and device IDs and drivers for "lspci -n" entries.
pci_id  dev_id    driver description
00:00.0 8086:0100 snb_uncore(module) "2nd Generation Core Processor Family DRAM Controller"
00:1A.0 8086:1C2D ehci-pci(builtin) "6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2"
00:1C.0 8086:1C10 pcieport(builtin) "6 Series/C200 Series Chipset Family PCI Express Root Port 1"
00:1C.4 8086:1C18 pcieport(builtin) "6 Series/C200 Series Chipset Family PCI Express Root Port 5"
00:1D.0 8086:1C26 ehci-pci(builtin) "6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1"
00:1E.0 8086:244E <none> "82801 PCI Bridge"
00:1F.0 8086:1C52 lpc_ich(module) "C202 Chipset LPC Controller"
00:1F.2 8086:1C02 ahci(module) "6 Series/C200 Series Chipset Family 6 port Desktop SATA AHCI Controller"
00:1F.3 8086:1C22 i2c_i801(module) "6 Series/C200 Series Chipset Family SMBus Controller"
02:00.0 14E4:165A tg3(module) "NetXtreme BCM5722 Gigabit Ethernet PCI Express"
03:03.0 102B:0532 mgag200(module) "MGA G200eW WPCM450"
Done
The revised script:

Code: Select all

#!/usr/bin/perl
#Name:    map_hardware_to_drivers.pl
#version: 1.0.0
#Date:    2020-03-12
#Author:  David Mathog
#
#  Map the pci hardware list to drivers for the running kernel
#
#  assumes wget, lspci, and access to the internet are available
#  If access is not available specify one parameter to the already downloaded
#  PCI IDs file.
#
# 1.0.0 2020-03-12, initial version
# 1.0.1 2020-03-23, builtin was improperly deduced from /sys, only uses 
#     builtin if there is NO module link.
use strict;
use warnings;
my $IDLIST="https://pci-ids.ucw.cz/v2.2/pci.ids";
my $real_filename="/tmp/pci.ids";
my $kernel=`uname -r`;
chomp $kernel;
my $MODALIAS="/lib/modules/$kernel/modules.alias";
my %dev_hash_longname;
my %dev_hash_module;
my %drv_hash_module;
my %pci_hash_builtin;
my %pci_hash_module;
my $count=0;
my $vendor="";
my $device="";
my $long_name="";
my $module_name="";
my $pci_id;
my $dev_id;
my $ignore;

my $num_args = $#ARGV;
if ($num_args != 0) {
    print 
     "Usage: map_hardware_to_drivers.pl PciIDsFile\n\n",
     "If internet access is available let PciIDsFile = \"-\" and it will be downloaded\n",
     "automatically.  Otherwise on some machine do:\n\n",
     "   wget -O PciIDsFile $IDLIST\n\n",
     "Then copy that file by some means to the target and specify\n",
     "the file name on the command line.\n";
    exit;
}
my $filename=$ARGV[0];



if($filename eq '-'){
   my $messages = `wget -q -O $real_filename $IDLIST 2>/dev/null`;
   if($messages){
      print
        "Some problem running:\n\n",
        "  wget -q -O $real_filename 2>/dev/null\n\n",
	"returned:\n",
	"$messages\n";
      exit;
   }
}
else {
   $real_filename = $filename;
}

#######################
open(FH, $real_filename) or die "could not open file $real_filename";

print "Checking this machine for drivers\n";
print "Getting Device ID list from $IDLIST\n";
# Syntax in this file:
#   #comment line
#   <blank lines>
#   vendor  vendor_name
#	device  device_name				<-- single tab
#		subvendor subdevice  subsystem_name	<-- two tabs
#   Only the vendor and device lines are retained
#(This will obtain all PCI ID -> long name mappings even if the current kernel does
#not have the information.)
while (my $line = <FH>){
    chomp $line;
    if(! $line){ next; }
    if(substr($line,0,1) eq "#"){ next; }
    if(substr($line,0,2) eq "\t\t"){ next; }
    if(substr($line,0,1) eq "\t"){ #device line
       $device = substr($line,1,4);
       $long_name = substr($line,7);
       $dev_id = $vendor . ":" . uc $device;
       $dev_hash_longname{$dev_id} = $long_name;
    }
    else { #vendor line
       $vendor = uc substr($line,0,4);
    }
}
close(FH);
#       my ($dev_type, $dev_id, $dev_driver) = split(/\s+/,$line);

#print "Getting pci IDs and long names from lspci\n";
#foreach my $line (`lspci`){
#    chomp $line;
#    my $starts = index($line," ");
#    my $pci_id = uc substr($line,0,$starts);
#    my $ln = substr($line,$starts+1);
#    $dev_hash_longname{$pci_id}=$ln;
#}

#######################
# get all the PCI ID to module mappings.
print "Getting module list from /sys/bus/pci/drivers\n";
foreach my $line (`find /sys/bus/pci/drivers`){
    chomp $line;
    my $starts = index($line,"module");
    if($starts != -1){
       my $delim = rindex(substr($line,0,$starts-1),"/");
       my $driver= substr($line,$delim+1,$starts - $delim - 2);
       # print "driver MODULE  >$driver<\n";
       $drv_hash_module{$driver}=$driver . "(module)";
    }
}
#######################
# get all the PCI ID to builtin mappings.
# This all PCI ID to device minus any which are already known to be modules.
print "Getting PCI ID to builtin mappings from /sys/bus/pci/drivers\n";
foreach my $line (`find /sys/bus/pci/drivers`){
    chomp $line;
    my $starts = index($line,"/0000:");
    if($starts != -1){
       my $pci_id = uc substr($line,$starts + 6);
       my $delim = rindex(substr($line,0,$starts-1),"/");
       my $driver= substr($line,$delim+1,$starts - $delim -1);
       $pci_hash_module{$pci_id}=$driver . "(module)";
       if(!defined($drv_hash_module{$driver})){
          # print "pci_id BUILTIN >$pci_id< driver >$driver<\n";
          $pci_hash_builtin{$pci_id}=$driver . "(builtin)";
       }
    }
}

#######################
# get all the PCI ID to module name mappings.  Ignore subvendor and subdevice information.
# This uses the "alias pci:" lines which have the syntax:
#   alias pci:vVENDORdDEVICEsvSUBVENDORsdSUBDEVICEscIGNOREiIGNORE module_name
# extract the VENDOR and DEVICE files to construct a PCI ID like 1234:ABCD
# Use that to store the module_name value in a hash.
#

open(FH, $MODALIAS) or die "could not open file $MODALIAS";
print "Getting DEV ID to kernel module mappings from $MODALIAS\n";

my $next_delim;
while (my $line = <FH>){
    chomp $line;
    if(substr($line,0,10) eq 'alias pci:'){
#       print "line: $line\n";
       if(substr($line,11,1) eq "*"){ next; }
       $next_delim=index($line,"d",11);
       if($next_delim == -1){ die "File $MODALIAS has pci: line with unexpected syntax: $line"; }
       $vendor = substr($line,15,4);
       $next_delim=index($line,"sv",20);
       if($next_delim == -1){ die("File $MODALIAS has pci: line with unexpected syntax: $line"); }
       $device = substr($line,24,4);
       $dev_id = $vendor . ":" . $device;
       $next_delim=index($line," ",28);
       if($next_delim == -1){ die "File $MODALIAS has pci: line with unexpected syntax: $line"; }
       $module_name = substr($line,$next_delim + 1);
       if(defined($dev_hash_module{$dev_id})){ next; } #subvendor/subdevice lines
       $dev_hash_module{$dev_id} = $module_name . "(module)";
       $count++;
#       print "  parsed $dev_id $module_name\n";
    }
}
close(FH);

print "Summarize pci and device IDs and drivers for \"lspci -n\" entries.\n";
print "pci_id  dev_id    driver description\n";
foreach my $line (`lspci -n`){
    chomp $line;
    ($pci_id, $ignore, $dev_id)= split /\s/,$line;
    $dev_id = uc $dev_id;
    $pci_id = uc $pci_id;
    # print "di $dev_id pi $pci_id line $line\n";
    my $description = (defined($dev_hash_longname{$dev_id}) ? $dev_hash_longname{$dev_id} : "unknown");
    my $module;
    my $dmodule = (defined($dev_hash_module{$dev_id}) ? $dev_hash_module{$dev_id} : "");
    my $builtin = (defined($pci_hash_builtin{$pci_id}) ? $pci_hash_builtin{$pci_id} : "");
    if(!$dmodule){
       if($builtin){
          $module = $builtin;
       }
       elsif (defined($pci_hash_module{$pci_id})){
          $module = $pci_hash_module{$pci_id};
       }
       else {
          $module = "<none>";
       }
    }
    else {
       if($builtin){
          $module .= "/" . $builtin;
       }
       else {
          $module = $dmodule;
       }
    }
    print "$pci_id $dev_id $module \"$description\"\n";
}
if($filename eq "-"){
   unlink($real_filename);
}
print "Done\n";

Post Reply