GL325 - Linux Performance Tuning and Analysis

Need something different? Guru Labs has other Linux and security courses and can build a custom course that covers exactly what you need!

This class meets the needs of advanced Linux Systems Administrators expanding their knowledge of core workings of Linux. Theories and methodologies are covered for tuning, monitoring performance, and analyzing the performance of the Linux operating system.

OBJECTIVES:

  • Learn principles and methods of system tuning
  • Understand Linux architecture
  • Use system and networking benchmark tools for understanding system capabilities
  • Diagnose complex performance and functional system problems
  • Learn to profile system hardware and applications
  • Analyze system performance to identify patterns
  • Learn methods of remote system monitoring

Current Version: D03

Prerequisites:

An intermediate knowledge of the Linux operating system is required. The equivalent knowledge can be found in the GL120 "Linux Fundamentals" and GL250 "Enterprise Linux Systems Administration" courses.

Supported Distributions:
Red Hat Enterprise Linux 7
Recommended Class Length:
4 days
Detailed Course Outline:
  1. Tuning Principles & The USE Method
    1. What is Performance Tuning?
    2. Performance Tuning Methodology
    3. Model System
    4. Establish a Baseline
    5. Establish Performance Targets
    6. Identify Bottleneck
    7. Tune
    8. Measure, Monitor, Evaluate, Repeat?
    9. Final Thoughts
    10. The USE Method
    11. Gathering USE Metrics
    Lab Tasks
    1. Determining the System's Configuration
    2. USE Method: CPU
    3. USE Method: Memory
    4. USE Method: Network
    5. USE Method: Storage I/O & Capacity
  2. Linux Internals Review
    1. The Linux Operating System Components
    2. Open Standards
    3. Memory Layout
    4. Important Kernel Threads
    5. Device Files in Linux
    6. Managing Linux Device Files
    7. udev
    8. Random Numbers and /dev/random
    9. Block Devices and the Device Mapper
    10. List Block Devices
    11. SCSI Devices
    12. File Systems with Linux
    13. Hardware Discovery Tools
    14. Cache and Write Barriers
    15. Advanced LVM: Components & Object Tags
    16. Advanced LVM: Automated Storage Tiering
    17. Advanced LVM: Thin Provisioning
    18. Advanced LVM: Striping & Mirroring
    19. Advanced LVM: RAID Volumes
    20. Kernel Modules
    Lab Tasks
    1. Adjusting Kernel Options
    2. Linux Kernel Driver Compilation
  3. Linux Userspace Architecture
    1. systemd System and Service Manager
    2. Using systemd
    3. Security Model Overview
    Lab Tasks
    1. Managing Services With Systemd's systemctl
    2. Setting Limits with the pam_limits Modules
    3. Operation of the setuid() and capset() System Calls
  4. Log File Administration
    1. System Logging
    2. systemd Journal
    3. systemd Journal's journalctl
    4. Secure Logging with Journal's Log Sealing
    5. gnome-system-log
    6. Rsyslog
    7. /etc/rsyslog.conf
    8. Log Management
    9. Log Anomaly Detector
    10. Sending logs from the shell
    Lab Tasks
    1. Using the systemd Journal
    2. Setting up a Full Debug Logfile
  5. Monitoring & Troubleshooting
    1. Viewing Processes
    2. System Status – Memory
    3. System Status – I/O
    4. System Status – CPU
    5. Performance Trending with sar
    6. Determining Service to Process Mapping
    7. Analyzing Process Memory Usage with pmap
    8. Real-time Monitoring of Resources — Cgroups
    9. strace and ltrace
    10. lsof and fuser
    11. ss/netstat and rpcinfo
    12. tcpdump and wireshark
    Lab Tasks
    1. Process Related Tools
    2. System Activity Reporter
    3. Cgroup for Processes
    4. Analyzing Memory Usage
  6. Benchmark Tools
    1. Benchmark Tools
    2. Disk/Filesystem Benchmarks
    3. Network Benchmarks
    4. Phoronix Test Suite
    Lab Tasks
    1. Benchmarking Disk and Filesystem
    2. Bonnie++ Visualization
  7. Systemtap
    1. SystemTap Overview
    2. SystemTap Features
    3. SystemTap Requirements
    4. Invoking SystemTap
    5. The 5 passes of stap
    6. SystemTap Probe Point Syntax
    7. SystemTap Probe Modes
    8. SystemTap Probe Points
    9. SystemTap process Probes
    10. Listing and Using Probe Points
    11. Built-in SystemTap Functionality
    12. SystemTap Example Repositories
    13. SystemTap Advanced Example
    14. SystemTap Documentation
    Lab Tasks
    1. System Tap Intro
    2. System Tap
    3. Creating SystemTap Scripts
    4. SystemTap One-liners & Histograms
  8. Performance Co-Pilot
    1. Performance Co-Pilot Introduction
    2. PCP Architecture
    3. PCP Performance Metrics
    4. PCP Installation
    5. PCP Performance Monitoring
    6. PCP Centralized Logging
    7. PCP Inference Engine
    8. PCP Deployment Strategies
    9. PCP Advanced Topics
    Lab Tasks
    1. Performance Co-Pilot Introduction
    2. Live Monitoring with pmchart
    3. Install Web Monitoring Tools
  9. Remote Metric Gathering With SNMP
    1. SNMP Overview
    2. SNMP Operation - Query
    3. MIBs and OID Namespace
    4. SNMP Security
    5. SNMP for Linux
    Lab Tasks
    1. SNMP
  10. Linux perf
    1. perf Overview
    2. perf Commands Typical Workflow
    3. perf Events
    4. perf stat
    5. perf record
    6. perf report
    Lab Tasks
    1. Perf
  11. Linux Tunables
    1. Operating System Tuning Caveats
    2. Consider OS (or Kernel) Upgrade
    3. Tuning CPU - NUMA
    4. Tuning CPU - Scheduling
    5. Huge Memory Pages
    6. Tuning Virtual Memory Manager
    7. Dropping Caches
    8. I/O Scheduler
    9. Tuning Filesystems
    10. Network Tuning
    11. tuned