Apache FastCGI Performance Probleme trotz 3 Servern verteilung.

AnnyKey · Feb 28, 2013

Hallo zusammen,

das Problem ist das meine Seite immer langsamer lädt und ich letzte Zeit mehr und mehr Besucher bekomme, im durchschnitt 1200 bis 1500 gleichzeitige Besucher, die pro 1 min. 1000-1200 Seitenaufrufe generieren was nicht stimmen kann, den mittags generieren 500 Besucher, 800-1000 Seitenaufrufe pro 1 min.
Die Auslastung des 1. Server wo das Script liegt ist bei 0.10 - 0.50 was bei dem 2. Server wo die mysql liegt, wesentlich höher ist zwischen 1.50 - 3.0.

Ich habe abends zwischen 17-23 Uhr ständig Probleme bei der Performance, die Webseite lädt sporadisch mal ganz schnell mal dauert es paar sec. genauso wie das Plesk Panel und manchmal trennt sich die Verbindung zwischen dem ssh, wenn ich dabei bin etwas zu konfigurieren etc.
In anderen Zeiten lädt sich die Seite ganz schnell, wie es sich auch gehört.
Ich hab jetzt meine Webseite auf 3 Servern verteilt und das Problem besteht weiterhin, ich hoffe mir kann jemand behilflich sein, hier die Auflistung & Konfiguration:

1. Server: Intel® Core™ i7-3770 CPU @ 3.40GHz, 24 GB DDR3 RAM, 2 x 3 TB SATA 6 Gb/s HDD 7200 rpm Software-RAID 1
für Bilder.

2. Server: Intel® Xeon® E3-1275 Quad-Core, 32 GB DDR3 RAM ECC, 1 x 256 GB SSD
für die Datenbank, mysql 5.5.30-1~dotdeb.0, Apache/2.2.16 (Debian), memchached.

3. Server: Intel® Core™ i7-3930K Hexa-Core, 64 GB DDR3 RAM, 1 x 256 GB SSD
für das Script.
Mit Plesk 11.0.9 (4 Domains), PHP 5.3, APC,
Apache2 Prefork,
OS Debian 6.0.7.

Aktivierte Module in Plesk beim 3. Server:
php5, auth_basic, authz_default, authz_groupfile, autoindex, cache, cgi, deflate, dir, expires, fcgid, headers, include, mem_cache, negotiation, reqtimeout, rewrite, rpaf, setenvif, status, suexec, userdir

fcgid.conf

Code:

FcgidProcessLifeTime 1000

# Maximum number of PHP processes.
FcgidMaxProcesses 2000
FcgidMaxProcessesPerClass 200
FcgidMinProcessesPerClass 0
FcgidMaxRequestsPerProcess 1000
FcgidMaxRequestLen 16777216
FcgidConnectTimeout 60

# Number of seconds of idle time before a process is terminated
FcgidIOTimeout 1000
FcgidIdleTimeout 240
FcgidInitialEnv RAILS_ENV production
FcgidIdleScanInterval 20

plesk.socket.sh

Code:

...
"max-procs" => 1,
"min-procs" => 1,
"idle-timeout" => 60,
"broken-scriptfilename" => "enable",
"allow-x-send-file" => "enable",
"bin-environment" => (
"PATH" => "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bi$
"PHP_FCGI_CHILDREN" => "16",
"PHP_FCGI_MAX_REQUESTS" => "5000"
...

apache2.conf

Code:

...
Timeout 5
KeepAlive On
MaxKeepAliveRequests 1000
KeepAliveTimeout 1
<IfModule mpm_prefork_module>
    ServerLimit          1500
    StartServers          200
    MinSpareServers        50
    MaxSpareServers       100
    MaxClients           1500
    MaxRequestsPerChild 90000
</IfModule>
HostnameLookups Off
LogLevel warn
GracefulShutDownTimeout 3
...

Ich frag mich ob die Servern ausreichend sind oder nicht und was man noch alles Optimieren bzw. Testen könnte?

d4f · Feb 28, 2013

Die Auslastung des 1. Server wo das Script liegt ist bei 0.10 - 0.50 was bei dem 2. Server wo die mysql liegt, wesentlich höher ist zwischen 1.50 - 3.0.

Die Load allein sagt nicht so viel aus, allerdings ist die Load generell "ok" solange sie < Anzahl der CPU-Threads ist.

nd manchmal trennt sich die Verbindung zwischen dem ssh

Ich würde spontan auf eine überfüllte Conntrack-Tabelle tippen. Sagt die kern.log oder syslog was spezifisches?

e Webseite lädt sporadisch mal ganz schnell mal dauert es paar sec.

Hast du keep-alive aktiviert?
Bei 2 Servern und erwarteten 3k Requests/Minute sollten zirka 5-6 gleichzeitige Requests auf Apache reinprasseln was definitiv kein Problem darstellen sollte.

Warum überhaupt fastcgi? Das ist bedeutend langsamer als mod_php wenn du schon auf Apache setzen willst.

und das Problem besteht weiterhin,

Von den Mountoptionen der Festplatte und Kernelparameter über Mysql-Einstellungen und Apache-Einstellungen bis zu deinen PHP-SKripten kann alles (oder besser gesagt: eine Mischung) davon Schuld sein. "Einfach so" ist das fast unmöglich zu pinpointen.

AnnyKey · Feb 28, 2013

d4f said:
Hast du keep-alive aktiviert?

keepAlive ist auf On

d4f said:
Warum überhaupt fastcgi? Das ist bedeutend langsamer als mod_php wenn du schon auf Apache setzen willst.

Ich benutze FastCGI weil ich daran schon gewohnt bin und davor nginx + FastCGI + php5-fpm mit ISPConfig 3 genutzt habe (wobei die Ladezeiten noch länger waren) und der umstieg zu mod_php ne gewisse Zeit in Anspruch nehmen wird.
Ist das nicht so, das man bei größeren Webseiten FastCGI anstatt mod_php einsetzen sollte?

d4f said:
Ich würde spontan auf eine überfüllte Conntrack-Tabelle tippen. Sagt die kern.log oder syslog was spezifisches?

Wie finde ich raus ob die Conntrack-Tabelle überfüllt ist?

Die kern.log sagt folgendes, Auszug von heute:

Code:

Feb 28 00:20:23 web kernel: [ 6277.273194] warning: `in.proftpd' uses 32-bit capabilities (legacy support in use)
Feb 28 12:29:31 web kernel: [50025.336207] sw-engine-cgi[1711]: segfault at 1000721 ip 00000000008ce059 sp 00007fff623bd810 error 4 in sw-engine-cgi[400000+ba4000]
Feb 28 12:29:37 web kernel: Kernel logging (proc) stopped.
Feb 28 12:30:33 web kernel: imklog 4.6.4, log source = /proc/kmsg started.
Feb 28 12:30:33 web kernel: [    0.000000] Initializing cgroup subsys cpuset
Feb 28 12:30:33 web kernel: [    0.000000] Initializing cgroup subsys cpu
Feb 28 12:30:33 web kernel: [    0.000000] Linux version 2.6.32-5-amd64 (Debian 2.6.32-48) (ben@decadent.org.uk) (gcc version 4.3.5 (Debian 4.3.5-4) ) #1 SMP Fri Feb 15 15:39:52 UTC 2013
Feb 28 12:30:33 web kernel: [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-2.6.32-5-amd64 root=UUID=7ad63505-f458-4fe0-b192-032f18e78985 ro nomodeset
Feb 28 12:30:33 web kernel: [    0.000000] KERNEL supported cpus:
Feb 28 12:30:33 web kernel: [    0.000000]   Intel GenuineIntel
Feb 28 12:30:33 web kernel: [    0.000000]   AMD AuthenticAMD
Feb 28 12:30:33 web kernel: [    0.000000]   Centaur CentaurHauls
Feb 28 12:30:33 web kernel: [    0.000000] BIOS-provided physical RAM map:
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 0000000000000000 - 000000000008f000 (usable)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 000000000008f000 - 00000000000a0000 (reserved)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 0000000000100000 - 00000000d8f7a000 (usable)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 00000000d8f7a000 - 00000000d915e000 (ACPI NVS)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 00000000d915e000 - 00000000d9457000 (usable)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 00000000d9457000 - 00000000d94d7000 (reserved)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 00000000d94d7000 - 00000000db6b9000 (usable)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 00000000db6b9000 - 00000000db6bf000 (reserved)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 00000000db6bf000 - 00000000db705000 (usable)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 00000000db705000 - 00000000db7bf000 (ACPI NVS)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 00000000db7bf000 - 00000000db7df000 (usable)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 00000000db7df000 - 00000000db7ff000 (ACPI data)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 00000000db7ff000 - 00000000db800000 (usable)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 00000000db800000 - 00000000e0000000 (reserved)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 00000000f0000000 - 0000000100000000 (reserved)
Feb 28 12:30:33 web kernel: [    0.000000]  BIOS-e820: 0000000100000000 - 0000001000000000 (usable)
Feb 28 12:30:33 web kernel: [    0.000000] DMI 2.6 present.
Feb 28 12:30:33 web kernel: [    0.000000] last_pfn = 0x1000000 max_arch_pfn = 0x400000000
Feb 28 12:30:33 web kernel: [    0.000000] MTRR default type: write-back
Feb 28 12:30:33 web kernel: [    0.000000] MTRR fixed ranges enabled:
Feb 28 12:30:33 web kernel: [    0.000000]   00000-9FFFF write-back
Feb 28 12:30:33 web kernel: [    0.000000]   A0000-BFFFF uncachable
Feb 28 12:30:33 web kernel: [    0.000000]   C0000-DFFFF write-protect
Feb 28 12:30:33 web kernel: [    0.000000]   E0000-FFFFF uncachable
Feb 28 12:30:33 web kernel: [    0.000000] MTRR variable ranges enabled:
Feb 28 12:30:33 web kernel: [    0.000000]   0 base 0000DC000000 mask 3FFFFC000000 uncachable
Feb 28 12:30:33 web kernel: [    0.000000]   1 base 0000E0000000 mask 3FFFF0000000 uncachable
Feb 28 12:30:33 web kernel: [    0.000000]   2 base 0000F0000000 mask 3FFFF8000000 uncachable
Feb 28 12:30:33 web kernel: [    0.000000]   3 base 0000F8000000 mask 3FFFFC000000 uncachable
Feb 28 12:30:33 web kernel: [    0.000000]   4 base 0000FC000000 mask 3FFFFE000000 uncachable
Feb 28 12:30:33 web kernel: [    0.000000]   5 base 0000FE000000 mask 3FFFFF000000 uncachable
Feb 28 12:30:33 web kernel: [    0.000000]   6 disabled
Feb 28 12:30:33 web kernel: [    0.000000]   7 disabled
Feb 28 12:30:33 web kernel: [    0.000000]   8 disabled
Feb 28 12:30:33 web kernel: [    0.000000]   9 disabled
Feb 28 12:30:33 web kernel: [    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
Feb 28 12:30:33 web kernel: [    0.000000] last_pfn = 0xdb800 max_arch_pfn = 0x400000000
Feb 28 12:30:33 web kernel: [    0.000000] initial memory mapped : 0 - 20000000
Feb 28 12:30:33 web kernel: [    0.000000] Using GB pages for direct mapping
Feb 28 12:30:33 web kernel: [    0.000000] init_memory_mapping: 0000000000000000-00000000db800000
Feb 28 12:30:33 web kernel: [    0.000000]  0000000000 - 00c0000000 page 1G
Feb 28 12:30:33 web kernel: [    0.000000]  00c0000000 - 00db800000 page 2M
Feb 28 12:30:33 web kernel: [    0.000000] kernel direct mapping tables up to db800000 @ 8000-a000
Feb 28 12:30:33 web kernel: [    0.000000] init_memory_mapping: 0000000100000000-0000001000000000
Feb 28 12:30:33 web kernel: [    0.000000]  0100000000 - 1000000000 page 1G
Feb 28 12:30:33 web kernel: [    0.000000] kernel direct mapping tables up to 1000000000 @ 9000-a000
Feb 28 12:30:33 web kernel: [    0.000000] RAMDISK: 375ba000 - 37fef7fd
Feb 28 12:30:33 web kernel: [    0.000000] ACPI: RSDP 00000000000fe020 00024 (v02 INTEL )
Feb 28 12:30:33 web kernel: [    0.000000] ACPI: XSDT 00000000db7fd120 0006C (v01 INTEL  DX79TO   000001CC      01000013)
Feb 28 12:30:33 web kernel: [    0.000000] ACPI: FACP 00000000db7fc000 000F4 (v03 INTEL  DX79TO   000001CC MSFT 0100000D)
Feb 28 12:30:33 web kernel: [    0.000000] ACPI: DSDT 00000000db7f7000 04F75 (v02 INTEL  DX79TO   000001CC MSFT 0100000D)
Feb 28 12:30:33 web kernel: [    0.000000] ACPI: FACS 00000000db70e000 00040
Feb 28 12:30:33 web kernel: [    0.000000] IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
Feb 28 12:30:33 web kernel: [    0.000000] ACPI: IOAPIC (id[0x01] address[0xfec3f000] gsi_base[24])
Feb 28 12:30:33 web kernel: [    0.000000] IOAPIC[1]: apic_id 1, version 32, address 0xfec3f000, GSI 24-47
Feb 28 12:30:33 web kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
Feb 28 12:30:33 web kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
Feb 28 12:30:33 web kernel: [    0.000000] ACPI: IRQ0 used by override.
Feb 28 12:30:33 web kernel: [    0.000000] ACPI: IRQ2 used by override.
Feb 28 12:30:33 web kernel: [    0.000000] ACPI: IRQ9 used by override.
Feb 28 12:30:33 web kernel: [    0.000000] Using ACPI (MADT) for SMP configuration information
Feb 28 12:30:33 web kernel: [    0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000
Feb 28 12:30:33 web kernel: [    0.000000] SMP: Allowing 16 CPUs, 4 hotplug CPUs
Feb 28 12:30:33 web kernel: [    0.000000] nr_irqs_gsi: 48
Feb 28 12:30:33 web kernel: [    0.000000] PM: Registered nosave memory: 000000000008f000 - 00000000000a0000
Feb 28 12:30:33 web kernel: [    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000e0000
Feb 28 12:30:33 web kernel: [    0.000000] PM: Registered nosave memory: 00000000000e0000 - 0000000000100000
Feb 28 12:30:33 web kernel: [    0.000000] PM: Registered nosave memory: 00000000d8f7a000 - 00000000d915e000
Feb 28 12:30:33 web kernel: [    0.000000] PM: Registered nosave memory: 00000000d9457000 - 00000000d94d7000
Feb 28 12:30:33 web kernel: [    0.000000] PM: Registered nosave memory: 00000000db6b9000 - 00000000db6bf000
Feb 28 12:30:33 web kernel: [    0.000000] PM: Registered nosave memory: 00000000db705000 - 00000000db7bf000
Feb 28 12:30:33 web kernel: [    0.000000] PM: Registered nosave memory: 00000000db7df000 - 00000000db7ff000
Feb 28 12:30:33 web kernel: [    0.000000] PM: Registered nosave memory: 00000000db800000 - 00000000e0000000
Feb 28 12:30:33 web kernel: [    0.000000] PM: Registered nosave memory: 00000000e0000000 - 00000000f0000000
Feb 28 12:30:33 web kernel: [    0.000000] PM: Registered nosave memory: 00000000f0000000 - 0000000100000000
Feb 28 12:30:33 web kernel: [    0.000000] Allocating PCI resources starting at e0000000 (gap: e0000000:10000000)
Feb 28 12:30:33 web kernel: [    0.000000] Booting paravirtualized kernel on bare hardware
Feb 28 12:30:33 web kernel: [    0.000000] NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:16 nr_node_ids:1
Feb 28 12:30:33 web kernel: [    0.000000] PERCPU: Embedded 30 pages/cpu @ffff880028200000 s90392 r8192 d24296 u131072
Feb 28 12:30:33 web kernel: [    0.000000] pcpu-alloc: s90392 r8192 d24296 u131072 alloc=1*2097152
Feb 28 12:30:33 web kernel: [    0.000000] pcpu-alloc: [0] 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 
Feb 28 12:30:33 web kernel: [    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 16397284
Feb 28 12:30:33 web kernel: [    0.000000] Policy zone: Normal
Feb 28 12:30:33 web kernel: [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-2.6.32-5-amd64 root=UUID=7ad63505-f458-4fe0-b192-032f18e78985 ro nomodeset
Feb 28 12:30:33 web kernel: [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
Feb 28 12:30:33 web kernel: [    0.000000] Initializing CPU#0
Feb 28 12:30:33 web kernel: [    0.000000] xsave/xrstor: enabled xstate_bv 0x7, cntxt size 0x340
Feb 28 12:30:33 web kernel: [    0.000000] Checking aperture...
Feb 28 12:30:33 web kernel: [    0.000000] No AGP bridge found
Feb 28 12:30:33 web kernel: [    0.000000] Calgary: detecting Calgary via BIOS EBDA area
Feb 28 12:30:33 web kernel: [    0.000000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
Feb 28 12:30:33 web kernel: [    0.000000] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Feb 28 12:30:33 web kernel: [    0.000000] Placing 64MB software IO TLB between ffff880020000000 - ffff880024000000
Feb 28 12:30:33 web kernel: [    0.000000] software IO TLB at phys 0x20000000 - 0x24000000
Feb 28 12:30:33 web kernel: [    0.000000] Memory: 65509880k/67108864k available (3086k kernel code, 601812k absent, 997172k reserved, 2045k data, 592k init)
Feb 28 12:30:33 web kernel: [    0.000000] SLUB: Genslabs=14, HWalign=64, Order=0-3, MinObjects=0, CPUs=16, Nodes=1
Feb 28 12:30:33 web kernel: [    0.000000] Hierarchical RCU implementation.
Feb 28 12:30:33 web kernel: [    0.000000] NR_IRQS:4352 nr_irqs:944
Feb 28 12:30:33 web kernel: [    0.000000] Extended CMOS year: 2000
Feb 28 12:30:33 web kernel: [    0.000000] Console: colour VGA+ 80x25
Feb 28 12:30:33 web kernel: [    0.000000] console [tty0] enabled
Feb 28 12:30:33 web kernel: [    0.000000] hpet clockevent registered
Feb 28 12:30:33 web kernel: [    0.000000] Fast TSC calibration using PIT
Feb 28 12:30:33 web kernel: [    0.004000] Detected 3200.482 MHz processor.
Feb 28 12:30:33 web kernel: [    0.000003] Calibrating delay loop (skipped), value calculated using timer frequency.. 6400.96 BogoMIPS (lpj=12801928)
Feb 28 12:30:33 web kernel: [    0.000178] Security Framework initialized
Feb 28 12:30:33 web kernel: [    0.000270] SELinux:  Disabled at boot.
Feb 28 12:30:33 web kernel: [    0.003328] Dentry cache hash table entries: 8388608 (order: 14, 67108864 bytes)
Feb 28 12:30:33 web kernel: [    0.013661] Inode-cache hash table entries: 4194304 (order: 13, 33554432 bytes)
Feb 28 12:30:33 web kernel: [    0.018111] Mount-cache hash table entries: 256
Feb 28 12:30:33 web kernel: [    0.018289] Initializing cgroup subsys ns
Feb 28 12:30:33 web kernel: [    0.018372] Initializing cgroup subsys cpuacct
Feb 28 12:30:33 web kernel: [    0.018455] Initializing cgroup subsys devices
Feb 28 12:30:33 web kernel: [    0.018537] Initializing cgroup subsys freezer
Feb 28 12:30:33 web kernel: [    0.018619] Initializing cgroup subsys net_cls
Feb 28 12:30:33 web kernel: [    0.018714] CPU: Physical Processor ID: 0
Feb 28 12:30:33 web kernel: [    0.018793] CPU: Processor Core ID: 0
Feb 28 12:30:33 web kernel: [    0.018876] CPU: L1 I cache: 32K, L1 D cache: 32K
Feb 28 12:30:33 web kernel: [    0.019019] CPU: L2 cache: 256K
Feb 28 12:30:33 web kernel: [    0.019099] CPU: L3 cache: 12288K
Feb 28 12:30:33 web kernel: [    0.019179] CPU 0/0x0 -> Node 0
Feb 28 12:30:33 web kernel: [    0.019260] mce: CPU supports 18 MCE banks
Feb 28 12:30:33 web kernel: [    0.028166] CPU0: Thermal monitoring enabled (TM1)
Feb 28 12:30:33 web kernel: [    0.028248] CPU 0 MCA banks CMCI:0 CMCI:1 CMCI:3 CMCI:5 CMCI:6 CMCI:7 CMCI:8 CMCI:9 CMCI:10 CMCI:11 SHD:12 SHD:13 SHD:14 SHD:15 SHD:16 SHD:17
Feb 28 12:30:33 web kernel: [    0.029436] using mwait in idle threads.
Feb 28 12:30:33 web kernel: [    0.029517] Performance Events: Nehalem/Corei7 events, Intel PMU driver.
Feb 28 12:30:33 web kernel: [    0.029729] ... version:                3
Feb 28 12:30:33 web kernel: [    0.029809] ... bit width:              48
Feb 28 12:30:33 web kernel: [    0.029890] ... generic registers:      4
Feb 28 12:30:33 web kernel: [    0.029970] ... value mask:             0000ffffffffffff
Feb 28 12:30:33 web kernel: [    0.030052] ... max period:             000000007fffffff
Feb 28 12:30:33 web kernel: [    0.030134] ... fixed-purpose events:   3
Feb 28 12:30:33 web kernel: [    0.030215] ... event mask:             000000070000000f
Feb 28 12:30:33 web kernel: [    0.031054] ACPI: Core revision 20090903
Feb 28 12:30:33 web kernel: [    0.052322] DMAR: Host address width 40
Feb 28 12:30:33 web kernel: [    0.052402] DMAR: DRHD base: 0x000000fe900000 flags: 0x1
Feb 28 12:30:33 web kernel: [    0.052489] IOMMU fe900000: ver 1:0 cap d2078c106f0462 ecap f020fe
Feb 28 12:30:33 web kernel: [    0.052573] DMAR: RMRR base: 0x000000d945c000 end: 0x000000d945cfff
Feb 28 12:30:33 web kernel: [    0.052657] DMAR: RMRR base: 0x000000d945f000 end: 0x000000d945ffff
Feb 28 12:30:33 web kernel: [    0.052741] DMAR: No ATSR found
Feb 28 12:30:33 web kernel: [    0.052827] Not enabling x2apic, Intr-remapping init failed.
Feb 28 12:30:33 web kernel: [    0.052911] Setting APIC routing to physical flat
Feb 28 12:30:33 web kernel: [    0.053407] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
Feb 28 12:30:33 web kernel: [    0.093053] CPU0: Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz stepping 07
Feb 28 12:30:33 web kernel: [    0.199673] Booting processor 1 APIC 0x1 ip 0x6000
Feb 28 12:30:33 web kernel: [    0.068080] Initializing CPU#1
Feb 28 12:30:33 web kernel: [    0.144903] CPU: Physical Processor ID: 0
Feb 28 12:30:33 web kernel: [    0.144904] CPU: Processor Core ID: 0
Feb 28 12:30:33 web kernel: [    0.144906] CPU: L1 I cache: 32K, L1 D cache: 32K
Feb 28 12:30:33 web kernel: [    0.144908] CPU: L2 cache: 256K
Feb 28 12:30:33 web kernel: [    0.144909] CPU: L3 cache: 12288K
Feb 28 12:30:33 web kernel: [    0.144911] CPU 1/0x1 -> Node 0
Feb 28 12:30:33 web kernel: [    0.144933] CPU1: Thermal monitoring enabled (TM1)
Feb 28 12:30:33 web kernel: [    0.144935] CPU 1 MCA banks SHD:0 SHD:1 SHD:3 SHD:5 SHD:6 SHD:7 SHD:8 SHD:9 SHD:10 SHD:11 SHD:12 SHD:13 SHD:14 SHD:15 SHD:16 SHD:17
Feb 28 12:30:33 web kernel: [    0.287391] CPU1: Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz stepping 07
Feb 28 12:30:33 web kernel: [    0.288349] checking TSC synchronization [CPU#0 -> CPU#1]:
Feb 28 12:30:33 web kernel: [    0.308437] Measured 457208174 cycles TSC warp between CPUs, turning off TSC clock.
Feb 28 12:30:33 web kernel: [    0.008000] Marking TSC unstable due to check_tsc_sync_source failed
Feb 28 12:30:33 web kernel: [    0.188069] Booting processor 2 APIC 0x2 ip 0x6000
Feb 28 12:30:33 web kernel: [    0.008000] CPU: Physical Processor ID: 0
Feb 28 12:30:33 web kernel: [    0.008000] CPU: Processor Core ID: 5
Feb 28 12:30:33 web kernel: [    0.008000] CPU: L1 I cache: 32K, L1 D cache: 32K
Feb 28 12:30:33 web kernel: [    0.008000] CPU: L2 cache: 256K
Feb 28 12:30:33 web kernel: [    0.008000] CPU: L3 cache: 12288K
Feb 28 12:30:33 web kernel: [    0.008000] CPU 11/0xb -> Node 0
Feb 28 12:30:33 web kernel: [    0.008000] CPU11: Thermal monitoring enabled (TM1)
Feb 28 12:30:33 web kernel: [    0.008000] CPU 11 MCA banks SHD:0 SHD:1 SHD:3 SHD:5 SHD:6 SHD:7 SHD:8 SHD:9 SHD:10 SHD:11 SHD:12 SHD:13 SHD:14 SHD:15 SHD:16 SHD:17
Feb 28 12:30:33 web kernel: [    1.108096] CPU11: Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz stepping 07
Feb 28 12:30:33 web kernel: [    1.109060] Brought up 12 CPUs
Feb 28 12:30:33 web kernel: [    1.109140] Total of 12 processors activated (76801.49 BogoMIPS).
Feb 28 12:30:33 web kernel: [    1.112047] CPU0 attaching sched-domain:
Feb 28 12:30:33 web kernel: [    1.112050]  domain 0: span 0-1 level SIBLING
Feb 28 12:30:33 web kernel: [    1.112051]   groups: group ffff88002820fae0 cpus 0 (cpu_power = 589) group ffff88002822fae0 cpus 1 (cpu_power = 589)
Feb 28 12:30:33 web kernel: [    1.112055]   domain 1: span 0-11 level MC
Feb 28 12:30:33 web kernel: [    1.112057]    groups: group ffff88002820fbf0 cpus 0-1 (cpu_power = 1178) group ffff88002824fbf0 cpus 2-3 (cpu_power = 1178) group ffff88002828fbf0 cpus 4-5 (cpu_power = 1178) group ffff8800282cfbf0 cpus 6-7 (cpu_power = 1178) group ffff88002830fbf0 cpus 8-9 (cpu_power = 1178) group ffff88002834fbf0 cpus 10-11 (cpu_power = 1178)
Feb 28 12:30:33 web kernel: [    1.112066] CPU1 attaching sched-domain:
Feb 28 12:30:33 web kernel: [    1.112067]  domain 0: span 0-1 level SIBLING
Feb 28 12:30:33 web kernel: [    1.112069]   groups: group ffff88002822fae0 cpus 1 (cpu_power = 589) group ffff88002820fae0 cpus 0 (cpu_power = 589)
Feb 28 12:30:33 web kernel: [    1.112072]   domain 1: span 0-11 level MC
Feb 28 12:30:33 web kernel: [    1.112073]    groups: group ffff88002820fbf0 cpus 0-1 (cpu_power = 1178) group ffff88002824fbf0 cpus 2-3 (cpu_power = 1178) group ffff88002828fbf0 cpus 4-5 (cpu_power = 1178) group ffff8800282cfbf0 cpus 6-7 (cpu_power = 1178) group ffff88002830fbf0 cpus 8-9 (cpu_power = 1178) group ffff88002834fbf0 cpus 10-11 (cpu_power = 1178)
Feb 28 12:30:33 web kernel: [    1.112082] CPU2 attaching sched-domain:
Feb 28 12:30:33 web kernel: [    1.112083]  domain 0: span 2-3 level SIBLING
Feb 28 12:30:33 web kernel: [    1.112084]   groups: group ffff88002824fae0 cpus 2 (cpu_power = 589) group ffff88002826fae0 cpus 3 (cpu_power = 589)
Feb 28 12:30:33 web kernel: [    1.112087]   domain 1: span 0-11 level MC
Feb 28 12:30:33 web kernel: [    1.112088]    groups: group ffff88002824fbf0 cpus 2-3 (cpu_power = 1178) group ffff88002828fbf0 cpus 4-5 (cpu_power = 1178) group ffff8800282cfbf0 cpus 6-7 (cpu_power = 1178) group ffff88002830fbf0 cpus 8-9 (cpu_power = 1178) group ffff88002834fbf0 cpus 10-11 (cpu_power = 1178) group ffff88002820fbf0 cpus 0-1 (cpu_power = 1178)
Feb 28 12:30:33 web kernel: [    1.112096] CPU3 attaching sched-domain:
Feb 28 12:30:33 web kernel: [    1.112097]  domain 0: span 2-3 level SIBLING
Feb 28 12:30:33 web kernel: [    1.112098]   groups: group ffff88002826fae0 cpus 3 (cpu_power = 589) group ffff88002824fae0 cpus 2 (cpu_power = 589)
Feb 28 12:30:33 web kernel: [    1.112102]   domain 1: span 0-11 level MC
Feb 28 12:30:33 web kernel: [    1.112103]    groups: group ffff88002824fbf0 cpus 2-3 (cpu_power = 1178) group ffff88002828fbf0 cpus 4-5 (cpu_power = 1178) group ffff8800282cfbf0 cpus 6-7 (cpu_power = 1178) group ffff88002830fbf0 cpus 8-9 (cpu_power = 1178) group ffff88002834fbf0 cpus 10-11 (cpu_power = 1178) group ffff88002820fbf0 cpus 0-1 (cpu_power = 1178)
Feb 28 12:30:33 web kernel: [    1.112111] CPU4 attaching sched-domain:
Feb 28 12:30:33 web kernel: [    1.112112]  domain 0: span 4-5 level SIBLING
Feb 28 12:30:33 web kernel: [    1.112113]   groups: group ffff88002828fae0 cpus 4 (cpu_power = 589) group ffff8800282afae0 cpus 5 (cpu_power = 589)
Feb 28 12:30:33 web kernel: [    1.112116]   domain 1: span 0-11 level MC
Feb 28 12:30:33 web kernel: [    1.112117]    groups: group ffff88002828fbf0 cpus 4-5 (cpu_power = 1178) group ffff8800282cfbf0 cpus 6-7 (cpu_power = 1178) group ffff88002830fbf0 cpus 8-9 (cpu_power = 1178) group ffff88002834fbf0 cpus 10-11 (cpu_power = 1178) group ffff88002820fbf0 cpus 0-1 (cpu_power = 1178) group ffff88002824fbf0 cpus 2-3 (cpu_power = 1178)
Feb 28 12:30:33 web kernel: [    1.112126] CPU5 attaching sched-domain:
Feb 28 12:30:33 web kernel: [    1.112127]  domain 0: span 4-5 level SIBLING
Feb 28 12:30:33 web kernel: [    1.112128]   groups: group ffff8800282afae0 cpus 5 (cpu_power = 589) group ffff88002828fae0 cpus 4 (cpu_power = 589)
Feb 28 12:30:33 web kernel: [    1.112131]   domain 1: span 0-11 level MC
Feb 28 12:30:33 web kernel: [    1.112132]    groups: group ffff88002828fbf0 cpus 4-5 (cpu_power = 1178) group ffff8800282cfbf0 cpus 6-7 (cpu_power = 1178) group ffff88002830fbf0 cpus 8-9 (cpu_power = 1178) group ffff88002834fbf0 cpus 10-11 (cpu_power = 1178) group ffff88002820fbf0 cpus 0-1 (cpu_power = 1178) group ffff88002824fbf0 cpus 2-3 (cpu_power = 1178)
Feb 28 12:30:33 web kernel: [    1.112140] CPU6 attaching sched-domain:
Feb 28 12:30:33 web kernel: [    1.112141]  domain 0: span 6-7 level SIBLING
Feb 28 12:30:33 web kernel: [    1.112142]   groups: group ffff8800282cfae0 cpus 6 (cpu_power = 589) group ffff8800282efae0 cpus 7 (cpu_power = 589)
Feb 28 12:30:33 web kernel: [    1.112146]   domain 1: span 0-11 level MC
Feb 28 12:30:33 web kernel: [    1.112147]    groups: group ffff8800282cfbf0 cpus 6-7 (cpu_power = 1178) group ffff88002830fbf0 cpus 8-9 (cpu_power = 1178) group ffff88002834fbf0 cpus 10-11 (cpu_power = 1178) group ffff88002820fbf0 cpus 0-1 (cpu_power = 1178) group ffff88002824fbf0 cpus 2-3 (cpu_power = 1178) group ffff88002828fbf0 cpus 4-5 (cpu_power = 1178)
Feb 28 12:30:33 web kernel: [    1.112155] CPU7 attaching sched-domain:
Feb 28 12:30:33 web kernel: [    1.112156]  domain 0: span 6-7 level SIBLING
Feb 28 12:30:33 web kernel: [    1.112157]   groups: group ffff8800282efae0 cpus 7 (cpu_power = 589) group ffff8800282cfae0 cpus 6 (cpu_power = 589)
Feb 28 12:30:33 web kernel: [    1.112160]   domain 1: span 0-11 level MC
Feb 28 12:30:33 web kernel: [    1.112161]    groups: group ffff8800282cfbf0 cpus 6-7 (cpu_power = 1178) group ffff88002830fbf0 cpus 8-9 (cpu_power = 1178) group ffff88002834fbf0 cpus 10-11 (cpu_power = 1178) group ffff88002820fbf0 cpus 0-1 (cpu_power = 1178) group ffff88002824fbf0 cpus 2-3 (cpu_power = 1178) group ffff88002828fbf0 cpus 4-5 (cpu_power = 1178)
Feb 28 12:30:33 web kernel: [    1.112170] CPU8 attaching sched-domain:
Feb 28 12:30:33 web kernel: [    1.112171]  domain 0: span 8-9 level SIBLING
Feb 28 12:30:33 web kernel: [    1.112172]   groups: group ffff88002830fae0 cpus 8 (cpu_power = 589) group ffff88002832fae0 cpus 9 (cpu_power = 589)
Feb 28 12:30:33 web kernel: [    1.112175]   domain 1: span 0-11 level MC
Feb 28 12:30:33 web kernel: [    1.112176]    groups: group ffff88002830fbf0 cpus 8-9 (cpu_power = 1178) group ffff88002834fbf0 cpus 10-11 (cpu_power = 1178) group ffff88002820fbf0 cpus 0-1 (cpu_power = 1178) group ffff88002824fbf0 cpus 2-3 (cpu_power = 1178) group ffff88002828fbf0 cpus 4-5 (cpu_power = 1178) group ffff8800282cfbf0 cpus 6-7 (cpu_power = 1178)
Feb 28 12:30:33 web kernel: [    1.112184] CPU9 attaching sched-domain:
Feb 28 12:30:33 web kernel: [    1.112185]  domain 0: span 8-9 level SIBLING
Feb 28 12:30:33 web kernel: [    1.112186]   groups: group ffff88002832fae0 cpus 9 (cpu_power = 589) group ffff88002830fae0 cpus 8 (cpu_power = 589)
Feb 28 12:30:33 web kernel: [    1.112190]   domain 1: span 0-11 level MC
Feb 28 12:30:33 web kernel: [    1.112191]    groups: group ffff88002830fbf0 cpus 8-9 (cpu_power = 1178) group ffff88002834fbf0 cpus 10-11 (cpu_power = 1178) group ffff88002820fbf0 cpus 0-1 (cpu_power = 1178) group ffff88002824fbf0 cpus 2-3 (cpu_power = 1178) group ffff88002828fbf0 cpus 4-5 (cpu_power = 1178) group ffff8800282cfbf0 cpus 6-7 (cpu_power = 1178)
Feb 28 12:30:33 web kernel: [    1.112199] CPU10 attaching sched-domain:
Feb 28 12:30:33 web kernel: [    1.112200]  domain 0: span 10-11 level SIBLING
Feb 28 12:30:33 web kernel: [    1.112201]   groups: group ffff88002834fae0 cpus 10 (cpu_power = 589) group ffff88002836fae0 cpus 11 (cpu_power = 589)
Feb 28 12:30:33 web kernel: [    1.112205]   domain 1: span 0-11 level MC
Feb 28 12:30:33 web kernel: [    1.112206]    groups: group ffff88002834fbf0 cpus 10-11 (cpu_power = 1178) group ffff88002820fbf0 cpus 0-1 (cpu_power = 1178) group ffff88002824fbf0 cpus 2-3 (cpu_power = 1178) group ffff88002828fbf0 cpus 4-5 (cpu_power = 1178) group ffff8800282cfbf0 cpus 6-7 (cpu_power = 1178) group ffff88002830fbf0 cpus 8-9 (cpu_power = 1178)
Feb 28 12:30:33 web kernel: [    1.112214] CPU11 attaching sched-domain:
Feb 28 12:30:33 web kernel: [    1.112215]  domain 0: span 10-11 level SIBLING
Feb 28 12:30:33 web kernel: [    1.112216]   groups: group ffff88002836fae0 cpus 11 (cpu_power = 589) group ffff88002834fae0 cpus 10 (cpu_power = 589)
Feb 28 12:30:33 web kernel: [    1.112219]   domain 1: span 0-11 level MC
Feb 28 12:30:33 web kernel: [    1.112220]    groups: group ffff88002834fbf0 cpus 10-11 (cpu_power = 1178) group ffff88002820fbf0 cpus 0-1 (cpu_power = 1178) group ffff88002824fbf0 cpus 2-3 (cpu_power = 1178) group ffff88002828fbf0 cpus 4-5 (cpu_power = 1178) group ffff8800282cfbf0 cpus 6-7 (cpu_power = 1178) group ffff88002830fbf0 cpus 8-9 (cpu_power = 1178)
Feb 28 12:30:33 web kernel: [    1.112371] devtmpfs: initialized
Feb 28 12:30:33 web kernel: [    1.117681] regulator: core version 0.5
Feb 28 12:30:33 web kernel: [    1.117681] NET: Registered protocol family 16
Feb 28 12:30:33 web kernel: [    1.117681] ACPI: bus type pci registered
Feb 28 12:30:33 web kernel: [    1.117681] PCI: MCFG configuration 0: base f0000000 segment 0 buses 0 - 63
Feb 28 12:30:33 web kernel: [    1.117681] PCI: MCFG area at f0000000 reserved in E820
Feb 28 12:30:33 web kernel: [    1.117912] PCI: Using MMCONFIG at f0000000 - f3ffffff
Feb 28 12:30:33 web kernel: [    1.117994] PCI: Using configuration type 1 for base access
Feb 28 12:30:33 web kernel: [    1.123649] bio: create slab <bio-0> at 0
Feb 28 12:30:33 web kernel: [    1.124442] ACPI: EC: Look up EC in DSDT
Feb 28 12:30:33 web kernel: [    1.135711] ACPI: Interpreter enabled
Feb 28 12:30:33 web kernel: [    1.135792] ACPI: (supports S0 S3 S4 S5)
Feb 28 12:30:33 web kernel: [    1.136149] ACPI: Using IOAPIC for interrupt routing
Feb 28 12:30:33 web kernel: [    1.139419] ACPI: No dock devices found.
Feb 28 12:30:33 web kernel: [    1.139757] ACPI: PCI Root Bridge [PCI0] (0000:00)
Feb 28 12:30:33 web kernel: [    1.139907] pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
Feb 28 12:30:33 web kernel: [    1.140007] pci 0000:00:00.0: PME# disabled
Feb 28 12:30:33 web kernel: [    1.140160] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
Feb 28 12:30:33 web kernel: [    1.140245] pci 0000:00:01.0: PME# disabled
Feb 28 12:30:33 web kernel: [    1.140400] pci 0000:00:01.1: PME# supported from D0 D3hot D3cold
Feb 28 12:30:33 web kernel: [    1.144006] pci 0000:00:01.1: PME# disabled
Feb 28 12:30:33 web kernel: [    1.144164] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold
Feb 28 12:30:33 web kernel: [    1.144250] pci 0000:00:02.0: PME# disabled
Feb 28 12:30:33 web kernel: [    1.144403] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold
Feb 28 12:30:33 web kernel: [    1.144489] pci 0000:00:03.0: PME# disabled
Feb 28 12:30:33 web kernel: [    1.144695] pci 0000:00:05.4: reg 10 32bit mmio: [0xec222000-0xec222fff]
Feb 28 12:30:33 web kernel: [    1.144813] pci 0000:00:11.0: PME# supported from D0 D3hot D3cold
Feb 28 12:30:33 web kernel: [    1.144898] pci 0000:00:11.0: PME# disabled
Feb 28 12:30:33 web kernel: [    1.145028] pci 0000:00:16.0: reg 10 64bit mmio: [0xe0000000-0xe000000f]
Feb 28 12:30:33 web kernel: [    1.164253] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
Feb 28 12:30:33 web kernel: [    1.164518] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P32_._PRT]
Feb 28 12:30:33 web kernel: [    1.164564] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX0._PRT]
Feb 28 12:30:33 web kernel: [    1.164625] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.EVRP._PRT]
Feb 28 12:30:33 web kernel: [    1.164664] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.JEX0._PRT]
Feb 28 12:30:33 web kernel: [    1.164709] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.JEX1._PRT]
Feb 28 12:30:33 web kernel: [    1.164752] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEG1._PRT]
Feb 28 12:30:33 web kernel: [    1.164795] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEG2._PRT]
Feb 28 12:30:33 web kernel: [    1.168341] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 7 9 10 *11 12)
Feb 28 12:30:33 web kernel: [    1.169131] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 7 9 10 11 12) *0, disabled.
Feb 28 12:30:33 web kernel: [    1.170058] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 7 9 10 *11 12)
Feb 28 12:30:33 web kernel: [    1.170843] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 7 9 10 11 12) *0, disabled.
Feb 28 12:30:33 web kernel: [    1.171768] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 7 *9 10 11 12)
Feb 28 12:30:33 web kernel: [    1.172530] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 7 9 10 11 12) *0, disabled.
Feb 28 12:30:33 web kernel: [    1.173454] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 7 9 10 11 12) *0, disabled.
Feb 28 12:30:33 web kernel: [    1.174386] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 7 9 *10 11 12)
Feb 28 12:30:33 web kernel: [    1.175168] vgaarb: device added: PCI:0000:06:00.0,decodes=io+mem,owns=io+mem,locks=none
Feb 28 12:30:33 web kernel: [    1.175266] vgaarb: loaded
Feb 28 12:30:33 web kernel: [    1.175352] PCI: Using ACPI for IRQ routing
Feb 28 12:30:33 web kernel: [    1.175352] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
Feb 28 12:30:33 web kernel: [    1.176274] hpet0: 8 comparators, 64-bit 14.318180 MHz counter
Feb 28 12:30:33 web kernel: [    1.178362] Switching to clocksource hpet
Feb 28 12:30:33 web kernel: [    1.178447] pnp: PnP ACPI init
Feb 28 12:30:33 web kernel: [    1.178447] ACPI: bus type pnp registered
Feb 28 12:30:33 web kernel: [    1.178447] pnp: PnP ACPI: found 12 devices
Feb 28 12:30:33 web kernel: [    1.178447] ACPI: ACPI bus type pnp unregistered
Feb 28 12:30:33 web kernel: [    1.178447] system 00:01: iomem range 0xf0000000-0x107ffffff could not be reserved
Feb 28 12:30:33 web kernel: [    1.178447] system 00:01: iomem range 0xf8000000-0xffffffff could not be reserved
Feb 28 12:30:33 web kernel: [    1.178447] system 00:06: ioport range 0x454-0x457 has been reserved
Feb 28 12:30:33 web kernel: [    1.178447] system 00:07: ioport range 0x500-0x57f has been reserved
Feb 28 12:30:33 web kernel: [    1.178447] system 00:07: ioport range 0x400-0x453 has been reserved
Feb 28 12:30:33 web kernel: [    1.178447] system 00:07: ioport range 0x458-0x47f has been reserved
Feb 28 12:30:33 web kernel: [    1.178488] system 00:07: ioport range 0x680-0x6ff has been reserved
Feb 28 12:30:33 web kernel: [    1.185918]   alloc irq_desc for 16 on node -1
Feb 28 12:30:33 web kernel: [    1.185919]   alloc kstat_irqs on node -1
Feb 28 12:30:33 web kernel: [    1.185922] pci 0000:00:11.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Feb 28 12:30:33 web kernel: [    1.186009] pci 0000:00:11.0: setting latency timer to 64
Feb 28 12:30:33 web kernel: [    1.186016] pci 0000:00:1c.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Feb 28 12:30:33 web kernel: [    1.186101] pci 0000:00:1c.0: setting latency timer to 64
Feb 28 12:30:33 web kernel: [    1.186106] pci 0000:00:1e.0: setting latency timer to 64
Feb 28 12:30:33 web kernel: [    1.186109] pci_bus 0000:00: resource 0 io:  [0x00-0xffff]
Feb 28 12:30:33 web kernel: [    1.186110] pci_bus 0000:00: resource 1 mem: [0x000000-0xffffffffffffffff]
Feb 28 12:30:33 web kernel: [    1.186112] pci_bus 0000:02: resource 1 mem: [0xec100000-0xec1fffff]
Feb 28 12:30:33 web kernel: [    1.186113] pci_bus 0000:06: resource 0 io:  [0x1000-0x1fff]
Feb 28 12:30:33 web kernel: [    1.186115] pci_bus 0000:06: resource 1 mem: [0xec000000-0xec0fffff]
Feb 28 12:30:33 web kernel: [    1.186116] pci_bus 0000:06: resource 2 pref mem [0xe8000000-0xebffffff]
Feb 28 12:30:33 web kernel: [    1.186117] pci_bus 0000:07: resource 3 io:  [0x00-0xffff]
Feb 28 12:30:33 web kernel: [    1.186119] pci_bus 0000:07: resource 4 mem: [0x000000-0xffffffffffffffff]
Feb 28 12:30:33 web kernel: [    1.186134] NET: Registered protocol family 2
Feb 28 12:30:33 web kernel: [    1.186453] IP route cache hash table entries: 524288 (order: 10, 4194304 bytes)
Feb 28 12:30:33 web kernel: [    1.187248] TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
Feb 28 12:30:33 web kernel: [    1.188316] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
Feb 28 12:30:33 web kernel: [    1.188501] TCP: Hash tables configured (established 524288 bind 65536)
Feb 28 12:30:33 web kernel: [    1.188587] TCP reno registered
Feb 28 12:30:33 web kernel: [    1.188750] NET: Registered protocol family 1
Feb 28 12:30:33 web kernel: [    1.188860] pci 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Feb 28 12:30:33 web kernel: [    1.204026] pci 0000:00:1a.0: PCI INT A disabled
Feb 28 12:30:33 web kernel: [    1.204133]   alloc irq_desc for 23 on node -1
Feb 28 12:30:33 web kernel: [    1.204134]   alloc kstat_irqs on node -1
Feb 28 12:30:33 web kernel: [    1.204139] pci 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
Feb 28 12:30:33 web kernel: [    1.220023] pci 0000:00:1d.0: PCI INT A disabled
Feb 28 12:30:33 web kernel: [    1.220130] pci 0000:02:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Feb 28 12:30:33 web kernel: [    1.229161] pci 0000:02:00.0: PCI INT A disabled
Feb 28 12:30:33 web kernel: [    1.229246] pci 0000:06:00.0: Boot video device
Feb 28 12:30:33 web kernel: [    1.229276] Unpacking initramfs...
Feb 28 12:30:33 web kernel: [    1.376949] Freeing initrd memory: 10453k freed
Feb 28 12:30:33 web kernel: [    1.378790] audit: initializing netlink socket (disabled)
Feb 28 12:30:33 web kernel: [    1.378880] type=2000 audit(1362051030.376:1): initialized
Feb 28 12:30:33 web kernel: [    1.384002] HugeTLB registered 2 MB page size, pre-allocated 0 pages
Feb 28 12:30:33 web kernel: [    1.385070] VFS: Disk quotas dquot_6.5.2
Feb 28 12:30:33 web kernel: [    1.385180] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
Feb 28 12:30:33 web kernel: [    1.385312] msgmni has been set to 32768
Feb 28 12:30:33 web kernel: [    1.385720] alg: No test for stdrng (krng)
Feb 28 12:30:33 web kernel: [    1.385831] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
Feb 28 12:30:33 web kernel: [    1.385929] io scheduler noop registered
Feb 28 12:30:33 web kernel: [    1.386009] io scheduler anticipatory registered
Feb 28 12:30:33 web kernel: [    1.386090] io scheduler deadline registered
Feb 28 12:30:33 web kernel: [    1.386201] io scheduler cfq registered (default)
Feb 28 12:30:33 web kernel: [    1.386369]   alloc irq_desc for 48 on node -1
Feb 28 12:30:33 web kernel: [    1.386370]   alloc kstat_irqs on node -1
Feb 28 12:30:33 web kernel: [    1.386378] pcieport 0000:00:01.0: irq 48 for MSI/MSI-X
Feb 28 12:30:33 web kernel: [    1.386383] pcieport 0000:00:01.0: setting latency timer to 64
Feb 28 12:30:33 web kernel: [    1.386476]   alloc irq_desc for 49 on node -1
Feb 28 12:30:33 web kernel: [    1.386476]   alloc kstat_irqs on node -1
Feb 28 12:30:33 web kernel: [    1.386481] pcieport 0000:00:01.1: irq 49 for MSI/MSI-X
Feb 28 12:30:33 web kernel: [    1.386486] pcieport 0000:00:01.1: setting latency timer to 64
Feb 28 12:30:33 web kernel: [    1.386576]   alloc irq_desc for 50 on node -1
Feb 28 12:30:33 web kernel: [    1.386577]   alloc kstat_irqs on node -1
Feb 28 12:30:33 web kernel: [    1.386582] pcieport 0000:00:02.0: irq 50 for MSI/MSI-X
Feb 28 12:30:33 web kernel: [    1.386586] pcieport 0000:00:02.0: setting latency timer to 64
Feb 28 12:30:33 web kernel: [    1.386677]   alloc irq_desc for 51 on node -1
Feb 28 12:30:33 web kernel: [    1.386678]   alloc kstat_irqs on node -1
Feb 28 12:30:33 web kernel: [    1.386682] pcieport 0000:00:03.0: irq 51 for MSI/MSI-X
Feb 28 12:30:33 web kernel: [    1.386687] pcieport 0000:00:03.0: setting latency timer to 64
Feb 28 12:30:33 web kernel: [    1.386790]   alloc irq_desc for 52 on node -1
Feb 28 12:30:33 web kernel: [    1.386791]   alloc kstat_irqs on node -1
Feb 28 12:30:33 web kernel: [    1.386797] pcieport 0000:00:11.0: irq 52 for MSI/MSI-X
Feb 28 12:30:33 web kernel: [    1.386804] pcieport 0000:00:11.0: setting latency timer to 64
Feb 28 12:30:33 web kernel: [    1.386900]   alloc irq_desc for 53 on node -1
Feb 28 12:30:33 web kernel: [    1.386901]   alloc kstat_irqs on node -1
Feb 28 12:30:33 web kernel: [    1.386906] pcieport 0000:00:1c.0: irq 53 for MSI/MSI-X
Feb 28 12:30:33 web kernel: [    1.387930] Linux agpgart interface v0.103
Feb 28 12:30:33 web kernel: [    1.388046] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
Feb 28 12:30:33 web kernel: [    1.388358] input: Macintosh mouse button emulation as /devices/virtual/input/input0
Feb 28 12:30:33 web kernel: [    1.388491] PNP: No PS/2 controller found. Probing ports directly.
Feb 28 12:30:33 web kernel: [    1.391641] serio: i8042 KBD port at 0x60,0x64 irq 1
Feb 28 12:30:33 web kernel: [    1.391725] serio: i8042 AUX port at 0x60,0x64 irq 12
Feb 28 12:30:33 web kernel: [    1.391845] mice: PS/2 mouse device common for all mice
Feb 28 12:30:33 web kernel: [    1.391954] rtc_cmos 00:03: RTC can wake from S4
Feb 28 12:30:33 web kernel: [    1.392065] rtc_cmos 00:03: rtc core: registered rtc_cmos as rtc0
Feb 28 12:30:33 web kernel: [    1.392172] rtc0: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
Feb 28 12:30:33 web kernel: [    1.392261] cpuidle: using governor ladder
Feb 28 12:30:33 web kernel: [    1.392342] cpuidle: using governor menu
Feb 28 12:30:33 web kernel: [    1.392424] No iBFT detected.
Feb 28 12:30:33 web kernel: [    1.392663] TCP cubic registered
Feb 28 12:30:33 web kernel: [    1.392805] NET: Registered protocol family 10
Feb 28 12:30:33 web kernel: [    1.393267] Mobile IPv6
Feb 28 12:30:33 web kernel: [    1.393347] NET: Registered protocol family 17
Feb 28 12:30:33 web kernel: [    1.393470] PM: Resume from disk failed.
Feb 28 12:30:33 web kernel: [    1.393474] registered taskstats version 1
Feb 28 12:30:33 web kernel: [    1.393949] rtc_cmos 00:03: setting system clock to 2013-02-28 11:30:31 UTC (1362051031)
Feb 28 12:30:33 web kernel: [    1.394083] Initalizing network drop monitor service
Feb 28 12:30:33 web kernel: [    1.394198] Freeing unused kernel memory: 592k freed
Feb 28 12:30:33 web kernel: [    1.394346] Write protecting the kernel read-only data: 4244k
Feb 28 12:30:33 web kernel: [    1.402960] udev[156]: starting version 164
Feb 28 12:30:33 web kernel: [    1.418692] usbcore: registered new interface driver usbfs
Feb 28 12:30:33 web kernel: [    1.418800] usbcore: registered new interface driver hub
Feb 28 12:30:33 web kernel: [    1.418916] usbcore: registered new device driver usb
Feb 28 12:30:33 web kernel: [    1.420392] e1000e: Intel(R) PRO/1000 Network Driver - 1.2.20-k2
Feb 28 12:30:33 web kernel: [    1.420477] e1000e: Copyright(c) 1999 - 2011 Intel Corporation.
Feb 28 12:30:33 web kernel: [    1.420587]   alloc irq_desc for 20 on node -1
Feb 28 12:30:33 web kernel: [    1.420589]   alloc kstat_irqs on node -1
Feb 28 12:30:33 web kernel: [    1.420594] e1000e 0000:00:19.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20
Feb 28 12:30:33 web kernel: [    1.420688] e1000e 0000:00:19.0: setting latency timer to 64
Feb 28 12:30:33 web kernel: [    1.420768]   alloc irq_desc for 54 on node -1
Feb 28 12:30:33 web kernel: [    1.420769]   alloc kstat_irqs on node -1
Feb 28 12:30:33 web kernel: [    1.420777] e1000e 0000:00:19.0: irq 54 for MSI/MSI-X
Feb 28 12:30:33 web kernel: [    1.426561] SCSI subsystem initialized
Feb 28 12:30:33 web kernel: [    1.426958] xhci_hcd 0000:02:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Feb 28 12:30:33 web kernel: [    1.427075] xhci_hcd 0000:02:00.0: setting latency timer to 64
Feb 28 12:30:33 web kernel: [    1.427078] xhci_hcd 0000:02:00.0: xHCI Host Controller
Feb 28 12:30:33 web kernel: [    1.427172] xhci_hcd 0000:02:00.0: new USB bus registered, assigned bus number 1
Feb 28 12:30:33 web kernel: [    1.427369] xhci_hcd 0000:02:00.0: irq 16, io mem 0xec100000
Feb 28 12:30:33 web kernel: [    1.427810] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
Feb 28 12:30:33 web kernel: [    1.430831] usb usb1: config 1 interface 0 altsetting 0 endpoint 0x81 has no SuperSpeed companion descriptor
Feb 28 12:30:33 web kernel: [    1.430937] usb usb1: New USB device found, idVendor=1d6b, idProduct=0003
Feb 28 12:30:33 web kernel: [    1.431021] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Feb 28 12:30:33 web kernel: [    1.431119] usb usb1: Product: xHCI Host Controller
Feb 28 12:30:33 web kernel: [    1.431200] usb usb1: Manufacturer: Linux 2.6.32-5-amd64 xhci_hcd
Feb 28 12:30:33 web kernel: [    1.431284] usb usb1: SerialNumber: 0000:02:00.0
Feb 28 12:30:33 web kernel: [    1.431410] usb usb1: configuration #1 chosen from 1 choice
Feb 28 12:30:33 web kernel: [    1.431493] xHCI xhci_add_endpoint called for root hub
Feb 28 12:30:33 web kernel: [    1.431494] xHCI xhci_check_bandwidth called for root hub
Feb 28 12:30:33 web kernel: [    1.431513] hub 1-0:1.0: USB hub found
Feb 28 12:30:33 web kernel: [    1.431597] hub 1-0:1.0: 4 ports detected
Feb 28 12:30:33 web kernel: [    1.435298] libata version 3.00 loaded.
Feb 28 12:30:33 web kernel: [    1.704650] e1000e 0000:00:19.0: eth0: (PCI Express:2.5GB/s:Width x1) 4c:72:b9:43:48:b3
Feb 28 12:30:33 web kernel: [    1.704749] e1000e 0000:00:19.0: eth0: Intel(R) PRO/1000 Network Connection
Feb 28 12:30:33 web kernel: [    1.704865] e1000e 0000:00:19.0: eth0: MAC: 10, PHY: 11, PBA No: FFFFFF-0FF
Feb 28 12:30:33 web kernel: [    1.704975] ehci_hcd 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Feb 28 12:30:33 web kernel: [    1.705096] ehci_hcd 0000:00:1a.0: setting latency timer to 64
Feb 28 12:30:33 web kernel: [    1.705098] ehci_hcd 0000:00:1a.0: EHCI Host Controller
Feb 28 12:30:33 web kernel: [    1.705188] ehci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 2
Feb 28 12:30:33 web kernel: [    1.705312] ehci_hcd 0000:00:1a.0: debug port 2
Feb 28 12:30:33 web kernel: [    1.709282] ehci_hcd 0000:00:1a.0: cache line size of 32 is not supported
Feb 28 12:30:33 web kernel: [    1.709286] ehci_hcd 0000:00:1a.0: irq 16, io mem 0xe0001000
Feb 28 12:30:33 web kernel: [    1.725391] ehci_hcd 0000:00:1a.0: USB 2.0 started, EHCI 1.00
Feb 28 12:30:33 web kernel: [    1.725499] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
Feb 28 12:30:33 web kernel: [    1.725585] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Feb 28 12:30:33 web kernel: [    1.734564] usb usb2: Product: EHCI Host Controller
Feb 28 12:30:33 web kernel: [    1.734646] usb usb2: Manufacturer: Linux 2.6.32-5-amd64 ehci_hcd
Feb 28 12:30:33 web kernel: [    1.734729] usb usb2: SerialNumber: 0000:00:1a.0
Feb 28 12:30:33 web kernel: [    1.734851] usb usb2: configuration #1 chosen from 1 choice
Feb 28 12:30:33 web kernel: [    1.734950] hub 2-0:1.0: USB hub found
Feb 28 12:30:33 web kernel: [    1.735034] hub 2-0:1.0: 2 ports detected
Feb 28 12:30:33 web kernel: [    1.735144] ahci 0000:00:1f.2: version 3.0
Feb 28 12:30:33 web kernel: [    1.735154]   alloc irq_desc for 18 on node -1
Feb 28 12:30:33 web kernel: [    1.735155]   alloc kstat_irqs on node -1
Feb 28 12:30:33 web kernel: [    1.735158] ahci 0000:00:1f.2: PCI INT C -> GSI 18 (level, low) -> IRQ 18
Feb 28 12:30:33 web kernel: [    1.735264]   alloc irq_desc for 55 on node -1
Feb 28 12:30:33 web kernel: [    1.735265]   alloc kstat_irqs on node -1
Feb 28 12:30:33 web kernel: [    1.735270] ahci 0000:00:1f.2: irq 55 for MSI/MSI-X
Feb 28 12:30:33 web kernel: [    1.735306] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x1 impl SATA mode
Feb 28 12:30:33 web kernel: [    1.735406] ahci 0000:00:1f.2: flags: 64bit ncq sntf pm led clo pio slum part ems apst 
Feb 28 12:30:33 web kernel: [    1.735506] ahci 0000:00:1f.2: setting latency timer to 64
Feb 28 12:30:33 web kernel: [    1.735562] scsi0 : ahci
Feb 28 12:30:33 web kernel: [    1.735707] scsi1 : ahci
Feb 28 12:30:33 web kernel: [    1.735836] scsi2 : ahci
Feb 28 12:30:33 web kernel: [    1.735962] scsi3 : ahci
Feb 28 12:30:33 web kernel: [    1.736096] scsi4 : ahci
Feb 28 12:30:33 web kernel: [    1.736226] scsi5 : ahci
Feb 28 12:30:33 web kernel: [    1.736407] ata1: SATA max UDMA/133 abar m2048@0xec223000 port 0xec223100 irq 55
Feb 28 12:30:33 web kernel: [    1.736505] ata2: DUMMY
Feb 28 12:30:33 web kernel: [    1.736583] ata3: DUMMY
Feb 28 12:30:33 web kernel: [    1.736660] ata4: DUMMY
Feb 28 12:30:33 web kernel: [    1.736739] ata5: DUMMY
Feb 28 12:30:33 web kernel: [    1.736817] ata6: DUMMY
Feb 28 12:30:33 web kernel: [    1.736913] ehci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
Feb 28 12:30:33 web kernel: [    1.737024] ehci_hcd 0000:00:1d.0: setting latency timer to 64
Feb 28 12:30:33 web kernel: [    1.737027] ehci_hcd 0000:00:1d.0: EHCI Host Controller
Feb 28 12:30:33 web kernel: [    1.737130] ehci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 3
Feb 28 12:30:33 web kernel: [    1.737253] ehci_hcd 0000:00:1d.0: debug port 2
Feb 28 12:30:33 web kernel: [    1.741204] ehci_hcd 0000:00:1d.0: cache line size of 32 is not supported
Feb 28 12:30:33 web kernel: [    1.741213] ehci_hcd 0000:00:1d.0: irq 23, io mem 0xe0002000
Feb 28 12:30:33 web kernel: [    1.757390] ehci_hcd 0000:00:1d.0: USB 2.0 started, EHCI 1.00
Feb 28 12:30:33 web kernel: [    1.757499] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002
Feb 28 12:30:33 web kernel: [    1.757584] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Feb 28 12:30:33 web kernel: [    1.757681] usb usb3: Product: EHCI Host Controller
Feb 28 12:30:33 web kernel: [    1.757763] usb usb3: Manufacturer: Linux 2.6.32-5-amd64 ehci_hcd
Feb 28 12:30:33 web kernel: [    1.757847] usb usb3: SerialNumber: 0000:00:1d.0
Feb 28 12:30:33 web kernel: [    1.757960] usb usb3: configuration #1 chosen from 1 choice
Feb 28 12:30:33 web kernel: [    1.758058] hub 3-0:1.0: USB hub found
Feb 28 12:30:33 web kernel: [    1.758141] hub 3-0:1.0: 2 ports detected
Feb 28 12:30:33 web kernel: [    2.044029] usb 2-1: new high speed USB device using ehci_hcd and address 2
Feb 28 12:30:33 web kernel: [    2.056031] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Feb 28 12:30:33 web kernel: [    2.057673] ata1.00: ATA-9: INTEL SSDSC2CW240A3, 400i, max UDMA/133
Feb 28 12:30:33 web kernel: [    2.057775] ata1.00: 468862128 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
Feb 28 12:30:33 web kernel: [    2.067686] ata1.00: configured for UDMA/133
Feb 28 12:30:33 web kernel: [    2.080122] scsi 0:0:0:0: Direct-Access     ATA      INTEL SSDSC2CW24 400i PQ: 0 ANSI: 5
Feb 28 12:30:33 web kernel: [    2.083299] sd 0:0:0:0: [sda] 468862128 512-byte logical blocks: (240 GB/223 GiB)
Feb 28 12:30:33 web kernel: [    2.083452] sd 0:0:0:0: [sda] Write Protect is off
Feb 28 12:30:33 web kernel: [    2.083535] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
Feb 28 12:30:33 web kernel: [    2.083557] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Feb 28 12:30:33 web kernel: [    2.083786]  sda: sda1 sda2 sda3
Feb 28 12:30:33 web kernel: [    2.084486] sd 0:0:0:0: [sda] Attached SCSI disk
Feb 28 12:30:33 web kernel: [    2.147145] md: linear personality registered for level -1
Feb 28 12:30:33 web kernel: [    2.148404] md: multipath personality registered for level -4
Feb 28 12:30:33 web kernel: [    2.149502] md: raid0 personality registered for level 0
Feb 28 12:30:33 web kernel: [    2.151200] md: raid1 personality registered for level 1
Feb 28 12:30:33 web kernel: [    2.152201] async_tx: api initialized (async)
Feb 28 12:30:33 web kernel: [    2.152553] xor: automatically using best checksumming function: generic_sse
Feb 28 12:30:33 web kernel: [    2.172631]    generic_sse: 14260.000 MB/sec
Feb 28 12:30:33 web kernel: [    2.172712] xor: using function: generic_sse (14260.000 MB/sec)
Feb 28 12:30:33 web kernel: [    2.180423] usb 2-1: New USB device found, idVendor=8087, idProduct=0024
Feb 28 12:30:33 web kernel: [    2.180509] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
Feb 28 12:30:33 web kernel: [    2.180633] usb 2-1: configuration #1 chosen from 1 choice
Feb 28 12:30:33 web kernel: [    2.180819] hub 2-1:1.0: USB hub found
Feb 28 12:30:33 web kernel: [    2.181047] hub 2-1:1.0: 6 ports detected
Feb 28 12:30:33 web kernel: [    2.240636] raid6: int64x1   3459 MB/s
Feb 28 12:30:33 web kernel: [    2.296012] usb 3-1: new high speed USB device using ehci_hcd and address 2
Feb 28 12:30:33 web kernel: [    2.308642] raid6: int64x2   3758 MB/s
Feb 28 12:30:33 web kernel: [    2.376641] raid6: int64x4   3396 MB/s
Feb 28 12:30:33 web kernel: [    2.428418] usb 3-1: New USB device found, idVendor=8087, idProduct=0024
Feb 28 12:30:33 web kernel: [    2.428502] usb 3-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
Feb 28 12:30:33 web kernel: [    2.428618] usb 3-1: configuration #1 chosen from 1 choice
Feb 28 12:30:33 web kernel: [    2.428813] hub 3-1:1.0: USB hub found
Feb 28 12:30:33 web kernel: [    2.429043] hub 3-1:1.0: 8 ports detected
Feb 28 12:30:33 web kernel: [    2.444647] raid6: int64x8   2977 MB/s
Feb 28 12:30:33 web kernel: [    2.504175] usb 2-1.4: new low speed USB device using ehci_hcd and address 3
Feb 28 12:30:33 web kernel: [    2.512636] raid6: sse2x1    8631 MB/s
Feb 28 12:30:33 web kernel: [    2.580635] raid6: sse2x2   10427 MB/s
Feb 28 12:30:33 web kernel: [    2.605548] usb 2-1.4: New USB device found, idVendor=0e8f, idProduct=0020
Feb 28 12:30:33 web kernel: [    2.605634] usb 2-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
Feb 28 12:30:33 web kernel: [    2.605730] usb 2-1.4: Product: PS2toUSB Adapter
Feb 28 12:30:33 web kernel: [    2.605810] usb 2-1.4: Manufacturer: GASIA
Feb 28 12:30:33 web kernel: [    2.605920] usb 2-1.4: configuration #1 chosen from 1 choice
Feb 28 12:30:33 web kernel: [    2.610361] usbcore: registered new interface driver hiddev
Feb 28 12:30:33 web kernel: [    2.612770] input: GASIA PS2toUSB Adapter as /devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.4/2-1.4:1.0/input/input1
Feb 28 12:30:33 web kernel: [    2.612918] generic-usb 0003:0E8F:0020.0001: input,hidraw0: USB HID v1.10 Keyboard [GASIA PS2toUSB Adapter] on usb-0000:00:1a.0-1.4/input0
Feb 28 12:30:33 web kernel: [    2.617801] input: GASIA PS2toUSB Adapter as /devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.4/2-1.4:1.1/input/input2
Feb 28 12:30:33 web kernel: [    2.617949] generic-usb 0003:0E8F:0020.0002: input,hidraw1: USB HID v1.10 Mouse [GASIA PS2toUSB Adapter] on usb-0000:00:1a.0-1.4/input1
Feb 28 12:30:33 web kernel: [    2.618064] usbcore: registered new interface driver usbhid
Feb 28 12:30:33 web kernel: [    2.618149] usbhid: v2.6:USB HID core driver
Feb 28 12:30:33 web kernel: [    2.648632] raid6: sse2x4   12007 MB/s
Feb 28 12:30:33 web kernel: [    2.648712] raid6: using algorithm sse2x4 (12007 MB/s)
Feb 28 12:30:33 web kernel: [    2.651024] md: raid6 personality registered for level 6
Feb 28 12:30:33 web kernel: [    2.651107] md: raid5 personality registered for level 5
Feb 28 12:30:33 web kernel: [    2.651188] md: raid4 personality registered for level 4
Feb 28 12:30:33 web kernel: [    2.654409] md: raid10 personality registered for level 10
Feb 28 12:30:33 web kernel: [    2.656422] mdadm: sending ioctl 1261 to a partition!
Feb 28 12:30:33 web kernel: [    2.656520] mdadm: sending ioctl 1261 to a partition!
Feb 28 12:30:33 web kernel: [    2.657671] mdadm: sending ioctl 1261 to a partition!
Feb 28 12:30:33 web kernel: [    2.657768] mdadm: sending ioctl 1261 to a partition!
Feb 28 12:30:33 web kernel: [    2.657950] mdadm: sending ioctl 800c0910 to a partition!
Feb 28 12:30:33 web kernel: [    2.658047] mdadm: sending ioctl 800c0910 to a partition!
Feb 28 12:30:33 web kernel: [    2.663908] device-mapper: uevent: version 1.0.3
Feb 28 12:30:33 web kernel: [    2.664073] device-mapper: ioctl: 4.15.0-ioctl (2009-04-01) initialised: dm-devel@redhat.com
Feb 28 12:30:33 web kernel: [    2.701947] EXT4-fs (sda3): mounted filesystem with ordered data mode
Feb 28 12:30:33 web kernel: [    2.768090] udev[490]: starting version 164
Feb 28 12:30:33 web kernel: [    2.786697] input: Sleep Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0E:00/input/input3
Feb 28 12:30:33 web kernel: [    2.786812] ACPI: Sleep Button [SLPB]
Feb 28 12:30:33 web kernel: [    2.787052] Monitor-Mwait will be used to enter C-1 state
Feb 28 12:30:33 web kernel: [    2.787092] processor LNXCPU:00: registered as cooling_device0
Feb 28 12:30:33 web kernel: [    2.787323] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input4
Feb 28 12:30:33 web kernel: [    2.787430] ACPI: Power Button [PWRF]
Feb 28 12:30:33 web kernel: [    2.787977] ACPI: WMI: Mapper loaded
Feb 28 12:30:33 web kernel: [    2.797760] processor LNXCPU:01: registered as cooling_device1
Feb 28 12:30:33 web kernel: [    2.798321] processor LNXCPU:02: registered as cooling_device2
Feb 28 12:30:33 web kernel: [    2.798809] processor LNXCPU:03: registered as cooling_device3
Feb 28 12:30:33 web kernel: [    2.799338] processor LNXCPU:04: registered as cooling_device4
Feb 28 12:30:33 web kernel: [    2.799556] input: PC Speaker as /devices/platform/pcspkr/input/input5
Feb 28 12:30:33 web kernel: [    2.799878] processor LNXCPU:05: registered as cooling_device5
Feb 28 12:30:33 web kernel: [    2.800573] processor LNXCPU:06: registered as cooling_device6
Feb 28 12:30:33 web kernel: [    2.801107] processor LNXCPU:07: registered as cooling_device7
Feb 28 12:30:33 web kernel: [    2.801618] processor LNXCPU:08: registered as cooling_device8
Feb 28 12:30:33 web kernel: [    2.802113] processor LNXCPU:09: registered as cooling_device9
Feb 28 12:30:33 web kernel: [    2.803057] processor LNXCPU:0a: registered as cooling_device10
Feb 28 12:30:33 web kernel: [    2.803628] processor LNXCPU:0b: registered as cooling_device11
Feb 28 12:30:33 web kernel: [    2.814941] Error: Driver 'pcspkr' is already registered, aborting...
Feb 28 12:30:33 web kernel: [    2.994471] Adding 33554424k swap on /dev/sda1.  Priority:-1 extents:1 across:33554424k SS
Feb 28 12:30:33 web kernel: [    3.005478] loop: module loaded
Feb 28 12:30:33 web kernel: [    3.066056] kjournald starting.  Commit interval 5 seconds
Feb 28 12:30:33 web kernel: [    3.066096] EXT3 FS on sda2, internal journal
Feb 28 12:30:33 web kernel: [    3.066098] EXT3-fs: mounted filesystem with ordered data mode.
Feb 28 12:30:33 web kernel: [    3.256145] e1000e 0000:00:19.0: irq 54 for MSI/MSI-X
Feb 28 12:30:33 web kernel: [    3.312068] e1000e 0000:00:19.0: irq 54 for MSI/MSI-X
Feb 28 12:30:33 web kernel: [    3.312909] ADDRCONF(NETDEV_UP): eth0: link is not ready
Feb 28 12:30:36 web kernel: [    6.167675] e1000e: eth0 NIC Link is Up 100 Mbps Half Duplex, Flow Control: Rx/Tx
Feb 28 12:30:36 web kernel: [    6.167776] e1000e 0000:00:19.0: eth0: 10/100 speed: disabling TSO
Feb 28 12:30:36 web kernel: [    6.171925] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Feb 28 12:30:46 web kernel: [   16.632016] eth0: no IPv6 routers present
Feb 28 14:20:49 web kernel: [ 6619.577974] warning: `in.proftpd' uses 32-bit capabilities (legacy support in use)
Feb 28 16:50:42 web kernel: [15612.572665] device eth0 entered promiscuous mode
Feb 28 16:52:40 web kernel: [15730.373148] device eth0 left promiscuous mode
Feb 28 16:52:47 web kernel: [15737.324661] device eth0 entered promiscuous mode
Feb 28 17:06:29 web kernel: [16560.209146] device eth0 left promiscuous mode

Marentis · Feb 28, 2013

Der Vorteil von Fastcgi liegt darin, dass man das wwwrun Problem nicht hat. Das ist eben besonders für Hoster interessant, bzw. wenn viele verschiedene Websiten mit jeweils unterschiedlichen Benutzern auf der Maschine laufen.
Für dich *dürfte* Fastcgi in Verbindung mit dem Apachen hier also kein Vorteil sein, dazu sollten sich aber lieber noch die Profis äußern.

d4f · Feb 28, 2013

Der Vorteil von Fastcgi liegt darin, dass man das wwwrun Problem nicht hat. Das ist eben besonders für Hoster interessant

Das muss ich aber mal unbedingt meinem PHP sagen, dass es gefälligst unter www-data zu laufen habe und nicht dem entsprechenden Account

Stichwort mpm_itk löst das Problem ohne den enormen Ressourcenaufwand. Aber selbst grosse Provider wollen ja oft lieber wwwrun als moderne Techniken...
Ich selbst kann mit Millionen an Requests pro Tag keinerlei Probleme mit mpm_itk feststellen, im Gegenteil die Zusatzfunktionen wie MaxClientsVhost sind äusserst nützlich und es klappt auf Anhieb mit mod_cgroup.

keepAlive ist auf On

Das ist vermutlich eines deiner Hauptprobleme. Apache hat, ausser unter mpm_event welches erst seit Apache 2.4 stable ist, nicht die Kapazität um Verbindungen offen zu lassen. Abhilfe bei bleibender oder sogar verbesserter keep-alive schafft ein Reverse-Proxy mit eventorientierter Programmierung wie Nginx.

Ich benutze FastCGI weil ich daran schon gewohnt bin und davor nginx + FastCGI + php5-fpm mit ISPConfig 3 genutzt habe (wobei die Ladezeiten noch länger waren) und der umstieg zu mod_php ne gewisse Zeit in Anspruch nehmen wird.

Generell sollte das bedeutend einfacher sein als in die andere Richtung. Aber mal als Frage; wieso verschwendest du Unmengen an Ressourcen auf einem Multi-Hosting Control panel wie IspConfig oder Plesk wenn nur eine einzige grosse Seite drauf läuft. Es manuell zu konfigurieren wäre stabiler und performanter.

Wie finde ich raus ob die Conntrack-Tabelle überfüllt ist?

Entsprechende Meldungen in der kern.log und/oder Syslog über gedroppte Verbindungen.

Generell würde ich anraten ein entsprechendes Gesuche im SSF auf zu geben um kostenpflichtig einen Sysadmin deine Server zu optimieren lassen. (Vermutlich ist eine der Optimierungen das Plesk runter fliegen soll =D )
Deine Erfahrungen mit hohen Besucherzahlen und den damit einhergehenden Problemen scheinen nicht sehr hoch zu sein weshalb gerade bei so gut besuchten Webseiten das schnell ein Problem werden kann.

AnnyKey · Feb 28, 2013

Marentis said:
Der Vorteil von Fastcgi liegt darin, dass man das wwwrun Problem nicht hat. Das ist eben besonders für Hoster interessant, bzw. wenn viele verschiedene Websiten mit jeweils unterschiedlichen Benutzern auf der Maschine laufen.
Für dich *dürfte* Fastcgi in Verbindung mit dem Apachen hier also kein Vorteil sein, dazu sollten sich aber lieber noch die Profis äußern.

Das habe ich auch schonmal irgendwo gelesen was mich jedoch nicht betrifft.

Ich nutze jetzt Plesk, dort sind 3 Varianten die man auswählen kann:
PHP-Unterstützung (ausführen als Apache, FastCGI und CGI)
Das habe ich jedoch noch nicht getestet was PHP-Unterstützung (ausführen als Apache) bedeuten soll, weis jemand da was?

Bei ISPConfig 3 waren es wesentlich mehr zur auswahl, jetzt spiel ich mit dem Gedanken doch wieder zu ISPConfig 3 rüberzugehen, jedoch anstatt nginx, apache + mod_php zu nutzen.

AnnyKey · Mar 4, 2013

Hab über das Wochenende mit keepAlive On & Off und mit mod_php (Bei Plesk Apache Modul) ausprobiert und hab festgestellt das es keine verbesserungen gab. Problem besteht weiterhin das ab ca. 1.000 gleichzeitigen Besuchern der Server sporadisch ausfällt bzw. die Seite lädt länger.

Folgendes habe ich noch zusätzlich optimiert:

Aufbau der Datenbankverbindung mit mysqli "Persistente Verbindung".

Code:

$db = @new mysqli('p:ip-Adresse', 'benutzer', 'passwort', 'dbName', 3306);

/etc/sysctl.conf

Code:

...
net.ipv4.tcp_rmem=4096 87380 2097152
net.ipv4.tcp_wmem=4096 65536 2097152
...
net.core.somaxconn=16384
net.core.rmem_max=2097152
net.core.wmem_max=2097152
kernel.shmall=134217728
kernel.shmmax=134217728

Aber mal als Frage; wieso verschwendest du Unmengen an Ressourcen auf einem Multi-Hosting Control panel wie IspConfig oder Plesk wenn nur eine einzige grosse Seite drauf läuft. Es manuell zu konfigurieren wäre stabiler und performanter.

Weil ich nicht nur eine Domain für meine Webseite habe, sondern für viele andere Ländern eine eigene Domain mit einer Datenbank, deshalb benutze ich jetzt Plesk und das Problem sollte eigentlich nicht am Plesk oder ISPConfig liegen oder liege ich da falsch?

Generell würde ich anraten ein entsprechendes Gesuche im SSF auf zu geben um kostenpflichtig einen Sysadmin deine Server zu optimieren lassen.

Das könnte ich machen nur ich sehe das nicht als eine dauerhafte Lösung, da ich mit dem Programmieren meiner Webseite fertig bin habe ich viel Zeit um mich mit dem Server zu beschäftigen, muss aber zu geben, das ich noch ein Anfänger bin und erst seit paar Monaten mich mit Server Konfiguration etc. beschäftige.

Nun habe ich rausgefunden das beim Plesk bereits mpm-prefork und mpm-worker etc. installiert ist, werde ich es morgen mit FastCGI + mpm worker versuchen, da ich grade mod_php verwende kam es zu einem Fehler beim anwenden von worker und mit PHP via FastCGI sollte es keine Probleme bereiten.

Code:

Starting web server: apache2[Mon Mar 04 18:44:15 2013] [crit] Apache is running a threaded MPM, but your PHP Module is not compiled to be threadsafe.  You need to recompile PHP.
Pre-configuration failed
Action 'start' failed.
The Apache error log may have more information.

Wenn jemand damit bereits Erfahrung gesammelt hat, würde ich mich über eure posts freuen.

d4f · Mar 5, 2013

Aufbau der Datenbankverbindung mit mysqli "Persistente Verbindung".

Vermutlich hast du dadurch die Performance eher degradiert da generell standardmässig UNIX-Sockets verwendet werden statt der volle TCP-Stack.

nur ich sehe das nicht als eine dauerhafte Lösung, da ich mit dem Programmieren meiner Webseite fertig bin habe ich viel Zeit um mich mit dem Server zu beschäftige

Selbst Jahre an Erfahrung helfen nicht unbedingt Engpässe zu lokalisieren und korrekt zu umgehen.
Aktuell hast du selbst keine Ahnung _was_ dich ausbremst - du kannst entweder wochenlang herumraten oder es schneller lösen. Je nachdem ob Geld oder Zeit dir wichtiger ist =)

sollte eigentlich nicht am Plesk oder ISPConfig liegen oder liege ich da falsch?

Eigentlich nicht, beide benötigen aber nennenswerte Leistung und greifen tief ins System ein. Zumal letzteres ist suboptimal da es den Freiraum für Anpassungen stark reduziert.

Nun habe ich rausgefunden das beim Plesk bereits mpm-prefork und mpm-worker etc. installiert ist

Ganz sicher nicht "und". Höchstens "oder". Beide sind prinzipbedingt nicht gleichzeitig einsetzbar.

und mit PHP via FastCGI sollte es keine Probleme bereiten.

Wie gesagt ist fastcgi nicht sehr leistungsstark. Wenn du etwas "besseres" als mod_php willst dann musst du zu fpm greifen.

PHP kann/soll nicht multithreaded werden, entsprechend ist es nur mit prefork und darauf basierenden MPM's kompatibel.

Was sagt denn eigentlich deine syslog und generell alle Logs zu den Problemzeitpunkten. Genau wie top, "atop 1" und iotop

Spin-Doc · Mar 5, 2013

d4f said:
Wie gesagt ist fastcgi nicht sehr leistungsstark. Wenn du etwas "besseres" als mod_php willst dann musst du zu fpm greifen.

Kannst du das auch irgendwie belegen? Meiner Erfahrung nach liefern mod_php, fcgid und fpm in etwa die gleiche Leistung. Wobei fcgid und fpm architektur-bedingt bei hoher Auslastung besser skaliert werden können.

d4f said:
PHP kann/soll nicht multithreaded werden, entsprechend ist es nur mit prefork und darauf basierenden MPM's kompatibel.

Dis gilt nur für mod_php. fcgid und fpm kann man sehr wohl z.B. mit dem worker mpm laufen lassen.

DerNeueDa · Mar 5, 2013

Ich hatte schon mal PHP als mod_php mit mpm_worker am laufen.. keine Probleme

Spin-Doc · Mar 5, 2013

worker + mod_php geht schon, ist aber nicht unbedingt sinnvoll, da nicht alle PHP Modulke threadsicher sind.

Meines Wissens nach unterbinden die meisten Distributionen eine Installation in der Konstellation mod_php + worker. Bei Debian zum. mault der Paket manager. Gentoo müsste das aber mit entsprechend gesetzten Flags zulassen (zum. ging das früher).

AnnyKey · Mar 5, 2013

d4f said:
Selbst Jahre an Erfahrung helfen nicht unbedingt Engpässe zu lokalisieren und korrekt zu umgehen.
Aktuell hast du selbst keine Ahnung _was_ dich ausbremst - du kannst entweder wochenlang herumraten oder es schneller lösen. Je nachdem ob Geld oder Zeit dir wichtiger ist =)

Beides ist mir wichtig, dann sag ich mal so, ich würde schon Geld ausgeben für Professionelles Serverwissen, jedoch aber nicht für eine Serverkonfiguration ohne das ich nicht weiss, was dort alles gemacht wurde.

d4f said:
Vermutlich hast du dadurch die Performance eher degradiert da generell standardmässig UNIX-Sockets verwendet werden statt der volle TCP-Stack.

Die Datenbank habe ich jetzt auf den selben Server übertragen wo das script liegt, dies hat sich positiv auf die Performance ausgewirkt und werde es heute mal mit und ohne p: bei mysqli beobachten.
Frage: Kann man das Problem lösen in dem man einen gleichen Server im gleichen RZ nutzt, da der jetztige Server (für mysql) im anderen RZ steht?

d4f said:
Ganz sicher nicht "und". Höchstens "oder". Beide sind prinzipbedingt nicht gleichzeitig einsetzbar.

Istalliert sind beide bei Plesk 11.0.9, ein Blick in das Paket apache2-bin zeigt, dass diverse MPMs direkt mitgeliefert werden. Diese befinden sich unter /usr/lib/apache2/. Ebenfalls enthalten sind dort: mpm-prefork und mpm-worker.

Dann noch folgendes Shell Kommando ausgeführt und so leicht war der umstieg auf mpm_worker, natürlich vorher mod_php deaktiviert, sonst geht das nicht:

Code:

/etc/init.d/apache2 stop
cd /usr/sbin
mv apache2 apache2prefork
ln -s ../lib/apache2/mpm-worker/apache2 apache2
/etc/init.d/apache2 start

d4f said:
Wie gesagt ist fastcgi nicht sehr leistungsstark. Wenn du etwas "besseres" als mod_php willst dann musst du zu fpm greifen.

Ich sah da keinen Unterschied zwischen mod_php und fastcgi, beide haben für die Performance nichts gebracht.

d4f said:
Was sagt denn eigentlich deine syslog und generell alle Logs zu den Problemzeitpunkten. Genau wie top, "atop 1" und iotop

Die apache error.log sagt folgendes:

Code:

...
[Mon Mar 04 20:28:33 2013] [notice] suEXEC mechanism enabled (wrapper: /usr/lib/apache2/suexec)
[Mon Mar 04 20:28:33 2013] [notice] Digest: generating secret for digest authentication ...
[Mon Mar 04 20:28:33 2013] [notice] Digest: done
[Mon Mar 04 20:28:34 2013] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Mon Mar 04 20:28:34 2013] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Mon Mar 04 20:28:34 2013] [warn] Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[Mon Mar 04 20:28:34 2013] [notice] Apache/2.2.16 (Debian) DAV/2 mod_fcgid/2.3.6 mod_ssl/2.2.16 OpenSSL/0.9.8o configured -- resuming normal operations
[Mon Mar 04 20:36:39 2013] [notice] caught SIGTERM, shutting down
[Mon Mar 04 20:36:40 2013] [notice] suEXEC mechanism enabled (wrapper: /usr/lib/apache2/suexec)
[Mon Mar 04 20:36:40 2013] [notice] Digest: generating secret for digest authentication ...
[Mon Mar 04 20:36:40 2013] [notice] Digest: done
[Mon Mar 04 21:18:59 2013] [error] [client **.***.**.**] request failed: error reading the headers, referer: http://domain.com/account/reply?id=14868056
...

top von heute, Server mit Script und Datenbank:

Code:

top - 16:53:14 up  3:52,  1 user,  load average: 3.98, 4.12, 4.21
Tasks: 371 total,   3 running, 368 sleeping,   0 stopped,   0 zombie
Cpu(s): 88.6%us,  1.7%sy,  0.0%ni,  9.3%id,  0.1%wa,  0.0%hi,  0.3%si,  0.0%st
Mem:  65520920k total, 27541564k used, 37979356k free,   236248k buffers
Swap: 33554424k total,        0k used, 33554424k free, 11624600k cached

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
1676 mysql     20   0 50.4g  12g 7904 S 1033 20.4   1131:12 mysqld
22653 web  20   0  314m  29m  19m S    4  0.0   0:00.34 php5-cgi
21076 webd  20   0  315m  27m  15m S    3  0.0   0:00.62 php5-cgi
22680 web  20   0  310m  19m  13m S    3  0.0   0:00.18 php5-cgi
22579 web  20   0  311m  21m  14m S    2  0.0   0:00.25 php5-cgi
22599 web  20   0  309m  17m  11m S    2  0.0   0:00.14 php5-cgi
22622 web  20   0  310m  18m  12m S    2  0.0   0:00.16 php5-cgi
22731 web  20   0  308m  18m  12m S    2  0.0   0:00.16 php5-cgi
15278 www-data  20   0 2286m  49m 2676 S    2  0.1   1:48.69 apache2
15281 www-data  20   0 2284m  44m 2400 S    2  0.1   1:48.58 apache2
22633 web  20   0  310m  22m  13m S    2  0.0   0:00.20 php5-cgi
22732 web  20   0  309m  18m  13m S    2  0.0   0:00.16 php5-cgi
15291 www-data  20   0 2283m  44m 2680 S    2  0.1   1:48.96 apache2
20980 web  20   0  313m  32m  22m S    2  0.1   0:01.69 php5-cgi
22637 web  20   0  312m  21m  12m S    2  0.0   0:00.18 php5-cgi
21035 web  20   0  314m  32m  21m S    1  0.1   0:01.05 php5-cgi
22623 web 20   0  312m  22m  13m S    1  0.0   0:00.22 php5-cgi
22697 web 20   0  312m  21m  12m S    1  0.0   0:00.17 php5-cgi
20778 web 20   0  314m  37m  26m S    1  0.1   0:02.07 php5-cgi
20885 web 20   0  314m  33m  23m S    1  0.1   0:01.32 php5-cgi
21000 web 20   0  310m  22m  15m S    1  0.0   0:00.50 php5-cgi
21058 web 20   0  312m  34m  25m S    1  0.1   0:02.08 php5-cgi
21061 web 20   0  312m  29m  20m S    1  0.0   0:00.71 php5-cgi
21062 web 20   0  310m  26m  20m R    1  0.0   0:00.96 php5-cgi
22588 web 20   0  310m  19m  13m S    1  0.0   0:00.22 php5-cgi
22646 web 20   0  309m  16m  10m S    1  0.0   0:00.10 php5-cgi
22661 web 20   0  310m  17m  10m S    1  0.0   0:00.10 php5-cgi
22754 web 20   0  310m  18m  12m S    1  0.0   0:00.15 php5-cgi
  431 root      20   0     0    0    0 S    1  0.0   0:19.00 jbd2/sda3-8
20882 web 20   0  316m  37m  25m S    1  0.1   0:01.39 php5-cgi
20953 web 20   0  312m  30m  21m S    1  0.0   0:00.86 php5-cgi
20983 web 20   0  312m  26m  17m S    1  0.0   0:00.90 php5-cgi
21033 web 20   0  312m  27m  19m S    1  0.0   0:00.65 php5-cgi
21052 web 20   0  312m  28m  20m S    1  0.0   0:01.05 php5-cgi
22651 web 20   0  310m  17m  11m R    1  0.0   0:00.16 php5-cgi
22716 web 20   0  308m  17m  11m S    1  0.0   0:00.10 php5-cgi
   45 root      20   0     0    0    0 S    0  0.0   0:02.17 events/6
 1201 root      20   0  114m 1680 1036 S    0  0.0   0:06.14 rsyslogd
 1719 root      20   0     0    0    0 S    0  0.0   0:14.55 flush-8:0
17813 www-data  20   0 31904 3240  900 S    0  0.0   0:09.51 nginx
17815 www-data  20   0 31916 3208  900 S    0  0.0   0:11.82 nginx
17817 www-data  20   0 32156 3388  900 S    0  0.0   0:09.60 nginx
17818 www-data  20   0 31948 3240  900 S    0  0.0   0:12.09 nginx

Die vhosts error.log und access.log habe ich deaktiviert, da dort nichts sinnvolles steht:

Code:

ErrorLog /dev/null
CustomLog /dev/null " " env=doesnt_exist

Heute habe ich zusätzlich Nginx als reverse proxy server vor apache konfiguriert und bis jetzt sieht es schon ganz anderst aus mit der Performance.
Apache (localhost) + Nginx als reverse proxy server + FastCGI + mpm_worker + APC + suexec, mem_cache habe ich deaktiviert.

Die apache2.conf sieht jetzt so aus:

Code:

Timeout 5
KeepAlive On
MaxKeepAliveRequests 1000
KeepAliveTimeout 1
<IfModule mpm_worker_module>
    StartServers          20
    MinSpareThreads      250
    MaxSpareThreads      750
    ThreadLimit          640
    ThreadsPerChild      250
    MaxClients          2500
    MaxRequestsPerChild    0
</IfModule>

Und die nginx.conf:

Code:

user www-data;
worker_processes  12;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
    worker_connections  2048;
    multi_accept on;
}

http {
    include       /etc/nginx/mime.types;

    access_log	/var/log/nginx/access.log;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  10;
    tcp_nodelay        on;

    gzip  on;
    gzip_disable "MSIE [1-6]\.(?!.*SV1)";
    gzip_comp_level 6;
    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

Die vhosts Konfiguration:

Code:

server {
    listen 80;
    server_name domain.com;
    root /var/www/vhosts/domain.com/httpdocs;

    location ~* ^.+\.(jpe?g|gif|png|ico|css|js|swf)$ {
    	expires max;
    }

    location / {
        proxy_pass http://127.0.0.1:8098;
    }
}

Mit den Konfigurationen bin ich noch nicht ganz zufrieden, weis jemand was man noch verbessern kann, besonderst beim nginx?

Spin-Doc · Mar 5, 2013

Ich würde mir erstmal dein mysql genauer ansehen, auf der ersten Blick ist das eher der Flaschenhals (Austabe top oben).

AnnyKey · Mar 5, 2013

Spin-Doc said:
Ich würde mir erstmal dein mysql genauer ansehen, auf der ersten Blick ist das eher der Flaschenhals (Austabe top oben).

Die sql Befehle habe ich jetzt untersucht und scheinen alle auser einen sehr schnell zu sein. Der eine dauert immer bei Spitzenzeiten ca. (25 insgesamt, Die Abfrage dauerte 2.6215 Sekunden) sonst bei 0.2-0.4, kann es nur daran liegen oder was könnte noch die ursache dafür sein bzw. was sollte man untersuchen?

Woran kann es genau bei mysql liegen das es VIRT 50.4g, RES 12g und die %CPU 1033 liegt?

Spin-Doc · Mar 5, 2013

Ich denke hier findest du ein paar Infos zu den üblichen Verdächtigen tuning-primer.sh und mysqltuner.pl

http://www.thomas-krenn.com/de/wiki/MySQL_Performance_Tuning

AnnyKey · Mar 5, 2013

Spin-Doc said:
Ich denke hier findest du ein paar Infos zu den üblichen Verdächtigen tuning-primer.sh und mysqltuner.pl

http://www.thomas-krenn.com/de/wiki/MySQL_Performance_Tuning

mysqltuner.pl und mysql-primer.sh sind bereits installiert.

Auszug aus mysqltuner.pl:

Code:

 >>  MySQLTuner 1.0.1 - Major Hayden <major@mhtx.net>
 >>  Bug reports, feature requests, and downloads at http://mysqltuner.com/
 >>  Run with '--help' for additional options and output filtering

-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.1.66-0+squeeze1
[OK] Operating on 64-bit architecture

-------- Storage Engine Statistics -------------------------------------------
[--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 48M (Tables: 110)
[--] Data in InnoDB tables: 4G (Tables: 216)
[--] Data in MEMORY tables: 4M (Tables: 6)
[!!] Total fragmented tables: 218

-------- Performance Metrics -------------------------------------------------
[--] Up for: 6h 46m 22s (31M q [1K qps], 155K conn, TX: 5B, RX: 2B)
[--] Reads / Writes: 98% / 2%
[--] Total buffers: 47.8G global + 10.2M per thread (3000 max threads)
[!!] Maximum possible memory usage: 77.6G (124% of installed RAM)
[OK] Slow queries: 0% (80K/31M)
[OK] Highest usage of available connections: 27% (827/3000)
[OK] Key buffer size / total MyISAM indexes: 256.0M/56.5M
[OK] Key buffer hit rate: 95.5% (6M cached / 314K reads)
[!!] Query cache efficiency: 10.7% (3M cached / 28M selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 2% (6K temp sorts / 295K sorts)
[!!] Temporary tables created on disk: 49% (212K on disk / 425K total)
[OK] Thread cache hit rate: 98% (1K created / 155K connections)
[OK] Table cache hit rate: 47% (1K open / 2K opened)
[OK] Open file limit used: 0% (267/68K)
[OK] Table locks acquired immediately: 99% (27M immediate / 27M locks)
[OK] InnoDB data size / buffer pool: 4.6G/43.0G

-------- Recommendations -----------------------------------------------------
General recommendations:
    Run OPTIMIZE TABLE to defragment tables for better performance
    MySQL started within last 24 hours - recommendations may be inaccurate
    Reduce your overall MySQL memory footprint for system stability
    Enable the slow query log to troubleshoot bad queries
    Temporary table size is already large - reduce result set size
    Reduce your SELECT DISTINCT queries without LIMIT clauses
Variables to adjust:
  *** MySQL's maximum memory usage is dangerously high ***
  *** Add RAM before increasing MySQL buffer variables ***
    query_cache_limit (> 512M, or use smaller result sets)

Auszug aus mysql-primer.sh:

Code:

        -- MYSQL PERFORMANCE TUNING PRIMER --
             - By: Matthew Montgomery -

MySQL Version 5.1.66-0+squeeze1 x86_64

Uptime = 0 days 6 hrs 46 min 44 sec
Avg. qps = 1295
Total Questions = 31611236
Threads Connected = 150

Warning: Server has not been running for at least 48hrs.
It may not be safe to use these recommendations

To find out more information on how each of these
runtime variables effects performance visit:
http://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html
Visit http://www.mysql.com/products/enterprise/advisors.html
for info about MySQL's Enterprise Monitoring and Advisory Service

SLOW QUERIES
The slow query log is NOT enabled.
Current long_query_time = 1.000000 sec.
You have 80313 out of 31611544 that take longer than 1.000000 sec. to complete
Your long_query_time seems to be fine

BINARY UPDATE LOG
The binary update log is NOT enabled.
You will not be able to do point in time recovery
See http://dev.mysql.com/doc/refman/5.1/en/point-in-time-recovery.html

WORKER THREADS
Current thread_cache_size = 100
Current threads_cached = 68
Current threads_per_sec = 0
Historic threads_per_sec = 0
Your thread_cache_size is fine

MAX CONNECTIONS
Current max_connections = 3000
Current threads_connected = 144
Historic max_used_connections = 827
The number of used connections is 27% of the configured maximum.
Your max_connections variable seems to be fine.

INNODB STATUS
Current InnoDB index space = 8.08 G
Current InnoDB data space = 4.64 G
Current InnoDB buffer pool free = 78 %
Current innodb_buffer_pool_size = 43.00 G
Depending on how much space your innodb indexes take up it may be safe
to increase this value to up to 2 / 3 of total system memory

MEMORY USAGE
Max Memory Ever Allocated : 53.97 G
Configured Max Per-thread Buffers : 29.84 G
Configured Max Global Buffers : 45.75 G
Configured Max Memory Limit : 75.59 G
Physical Memory : 62.48 G

Max memory limit exceeds 90% of physical memory

KEY BUFFER
Current MyISAM index space = 56 M
Current key_buffer_size = 256 M
Key cache miss rate is 1 : 21
Key buffer free ratio = 80 %
Your key_buffer_size seems to be fine

QUERY CACHE
Query cache is enabled
Current query_cache_size = 2.00 G
Current query_cache_used = 5 M
Current query_cache_limit = 512 M
Current Query cache Memory fill ratio = .24 %
Current query_cache_min_res_unit = 4 K
Your query_cache_size seems to be too high.
Perhaps you can use these resources elsewhere
MySQL won't cache query results that are larger than query_cache_limit in size

SORT OPERATIONS
Current sort_buffer_size = 2 M
Current read_rnd_buffer_size = 1 M
Sort buffer seems to be fine

JOINS
Current join_buffer_size = 6.00 M
You have had 3 queries where a join could not use an index properly
join_buffer_size >= 4 M
This is not advised
You should enable "log-queries-not-using-indexes"
Then look for non indexed joins in the slow query log.

OPEN FILES LIMIT
Current open_files_limit = 68546 files
The open_files_limit should typically be set to at least 2x-3x
that of table_cache if you have heavy MyISAM usage.
Your open_files_limit value seems to be fine

TABLE CACHE
Current table_open_cache = 32768 tables
Current table_definition_cache = 16384 tables
You have a total of 355 tables
You have 1202 open tables.
The table_cache value seems to be fine

TEMP TABLES
Current max_heap_table_size = 2.00 G
Current tmp_table_size = 2.00 G
Of 213238 temp tables, 49% were created on disk
Perhaps you should increase your tmp_table_size and/or max_heap_table_size
to reduce the number of disk-based temporary tables
Note! BLOB and TEXT columns are not allow in memory tables.
If you are using these columns raising these values might not impact your
ratio of on disk temp tables.

TABLE SCANS
Current read_buffer_size = 1 M
Current table scan ratio = 24 : 1
read_buffer_size seems to be fine

TABLE LOCKING
Current Lock Wait ratio = 1 : 628114
Your table locking seems to be fine

Wäre nett wenn jemand damit schon Erfahrung gesammelt hat und mir paar Ratschläge/Tipps geben kann.

Spin-Doc · Mar 6, 2013

Den langsamen Query optimieren sollte einiges bringen. Dann kannst du ja auch mal das slow query log anmachen, und schauen was sich dort noch alles findet.

Temporary tables created on disk ist recht hoch, evtl. kannst du dort Abhilfe schaffen. Die max. mögliche Speicherzuweisung ist natürlich auch etwas hoch. Wenn dir die Kiste irgendwann anfängt zu swappen dann geht der Performence natürlich in den Keller.

Bessere wäre es natürlich, dass Ding mal min. 24Std am Stück laufen zu lassen, und sich dann die Werte erneut anzusehen, weil so auf den ersten Blick schauts ansonsten doch recht gesund aus.

Apache FastCGI Performance Probleme trotz 3 Servern verteilung.

AnnyKey

New Member

d4f

Kaffee? Wo?

AnnyKey

New Member

Marentis

Guest

d4f

Kaffee? Wo?

AnnyKey

New Member

AnnyKey

New Member

d4f

Kaffee? Wo?

Spin-Doc

Registered User

DerNeueDa

New Member

Spin-Doc

Registered User

AnnyKey

New Member

Spin-Doc

Registered User

AnnyKey

New Member

Spin-Doc

Registered User

AnnyKey

New Member

Spin-Doc

Registered User

We value your privacy