WordPress log files are an excellent tool for understanding what goes on your websites. WordPress, however, doesn’t operate in a vacuum – it depends on several systems and sub-systems, all of which generate their own logs. Log management systems collate your WordPress activity log and other logs in one central location, providing unprecedented views of what happens on your WordPress site and the infrastructure surrounding it.
From spotting suspicious behavior to monitoring resources and improving user management, a Log Management System can help you better understand everything that goes on your websites and network. It also allows you to prevent issues and track back when things go wrong, among many other benefits.
This blog post looks into how a WordPress Log Management System can help you improve your WordPress website’s compliance, troubleshooting, and management.
Table of contents
What is a log file?
A log file is a record of events that take place on a system be it the server, website, plugin, app, or other piece of technology. Not all logging systems track all events. For example, WordPress can log errors out of the box but needs WP Activity Log to track system and user activity.
Generally speaking, logs record:
- Everyday events such as user logins and access requests
- Abnormal events such as errors, multiple login attempts by the same user, or breaches
Regardless of which type of event is being recorded, each log entry in a log file will typically be:
- Computer-generated and automatically produced when certain events occur
- Time-stamped
- Inclusive of details about the event, such as associated codes, user and device ID, IP address, and other useful information
Log files come in all shapes and sizes. A user activity log varies significantly from a firewall log, which also varies from the webserver log. Yet, all of these logs play their part, as we shall see in the next section.
What is log management?
In medium to large networks, administrators often find themselves dealing with log files from disparate systems. These logs are a valuable resource that plays a critical role in the execution of various tasks, from improving system administration and performance to WordPress GDPR compliance, security, and everything in between. As such, you should make sure that logs are carefully managed. This is where log management comes in.
The log management process defines how several tasks related to logs are executed, including:
- Collection – Defines how logs are collected and parsed
- Storage – Defines how and where logs are stored, compressed, archived, or deleted
- Analysis – Defines how logs are used and analyzed, including alerts and reports
Why is log management important?
Log management is particularly important for website administrators who manage larger systems. It provides a framework that ensures proper collection, storage, and analysis of logs. In turn, this ensures that the log value is maximized. For example, it ensures that logs are kept in check, are available when needed, and can provide actionable insight – among other things.
When using a log management system, logs are collated in one central place, where they are easier to manage and use. It also enables you to carry out deeper and wider analysis across different systems with far greater accuracy and in less time.
ISO 27001 Annex: A.12.4. Logging and monitoring standards
Logging and Monitoring is addressed in one of the most widely adopted international standards on information security (ISO/IEC 27001). In A.12.4.1, you can read about its recommendations on Event Logging. The standard recommends Event Logs should be retained to include items such as:
- The ID of the user, system activities, and key event details (log-on and log-off)
- Records of access attempts, including rejected ones
- Alterations in system configurations
- Accessed files
- Entry management system warnings
- Transaction records
Other compliance standards
There are many official compliance standards (NIST) and regulations (GDPR, GLBA, FISMA, HIPAA, SOX) that can inform your log management policy. Which one you employ depends on your business, location, and sector. Here are some important examples:
- Logging systems and mechanisms are a PCI DSS compliance stipulation for WordPress e-commerce websites such as WooCommerce
- NIST 800-53 is a cybersecurity compliance framework for federal institutions in North America. Logging requirements are outlined in Special Publication (SP) 800-82
- In the UK, the Ministry of Justice and the National Cyber Security Centre have similar guidelines for logging
For further information, see What is regulatory compliance & how does it affect WordPress security?
Log Management Systems
A Log Management System (LMS) is a repository that allows organizations and individuals to automatically collate individual logs from several systems into one central, convenient location. Administrators can then log into this system to gain an overview of their entire IT environment.
What functions does a log management system perform?
There are several main functions, each of which is a step in the log management process. These steps cover the entire journey of an event log from initial generation and parsing, through transmission and storage, to final disposing. They are:
- Centralizing – All the log files are collected from multiple sources and aggregated into one location with rules on how long this log data should be retained before it is archived or destroyed
- Standardizing – All the log data must be arranged – parsed, normalized (converted into a standard format), and indexed for monitoring and analysis
- Monitoring – All new log data from events and activities must be monitored, with alerts set for specific events that require attention
Once you centralized all of your log files in one Log Management System, you can begin analyzing and correlating the data. You can search through it and filter in multiple ways. This enables you to start making connections between events on different systems and build a more accurate picture of what is going on in your infrastructure.
What type of logs can you add to log management systems?
Log Management Systems typically enjoy wide compatibility with many different systems. However, before making any commitment to any one solution, you will need to do your research to ensure compatibility with the different systems in your infrastructure. Loggly, for example, lists all compatible systems on their website, making it easy to ensure you can integrate all of your systems.
Pay attention to the log formats of each system you want to integrate and compare it with the log file format that the Log Management System expects. Port and protocol are two other important factors you need to consider to make sure your implementation is a success.
Web server logs
WordPress runs on a web server. As such, it makes sense to collect logs from the WordPress web server as well. Apache is the most common web server; other web servers, such as Nginx and IIS, also enjoy considerable popularity. It’s also worth noting that WordPress web server logs typically include an error log file and an access log file, among others.
WordPress log
While WordPress does not have an out-of-the-box activity log, it does have an error/debug log. You’ll need to enable it through the wp_debug option since it’s switched off by default. You’ll also need to ensure you enable the WP_DEBUG_LOG to log errors to a log file. The log file can then be exported.
Firewall logs
Administrators deploy firewalls to protect WordPress sites as well as entire networks. Their logs usually include information on the port, protocol, IP address, and status, among other things. This can help you identify malicious attacks or behavior and devise ways to defend against them. Firewall logs can be especially useful when investigating security incidents.
WordPress activity logs
WordPress activity logs, also known as audit logs or security logs, track user and system activities across your WordPress website. You’ll need to install the WP Activity Log plugin, which you can easily do through the WordPress dashboard. This user and system activity tracking plugin offers comprehensive logging capabilities with support for 3rd party plugins, including WooCommerce, among many others.
Log management systems and WordPress activity logs
As we’ve already mentioned, WordPress does not keep an activity log out of the box. However, you can easily add this functionality by installing WP Activity Log. This plugin provides comprehensive logging capabilities and supports integration with many log management systems.
Once you install and activate the WP Activity Log, it automatically starts logging user activities and changes in your WordPress sites, multisite network, or eCommerce store. By default, the plugin stores the logs in the local WordPress database. However, you can integrate WP Activity Log with a number of Log Management Systems.
The plugin offers out-of-the-box integration with the following Log Management Systems:
- AWS CloudWatch
- Loggly
- Papertrail
- Syslog Server
WP Activity Log can also mirror the log to a log file, which outputs a .log file – a format commonly used by many Log Management Systems. It is also worth noting that WP Activity Log also supports connections to Slack and external MySQL databases.
The process of integrating WordPress activity logs with Log Management Systems is very straightforward and does not require any special technical skills. There is plenty of documentation available, and there is email support should you need additional help.
What does WP Activity Log monitor?
WP Activity Log is the most comprehensive activity log plugin available. It keeps a detailed activity log that includes:
- ID for easy referencing
- Severity level
- Date and time
- User undertaking the activity
- The IP address of the user
- The object that was modified
- The type of event
- Message with detailed information about the activity
You can send all of this information to the Log Management System. There, it can be referenced against activities from other systems, devices, and appliances.
The plugin offers extensive user activity tracking across WordPress and third-party plugins, with support for popular WordPress plugins available. WP Activity Log also includes a user sessions module, which enables you to manage user sessions in real time.
Benefits of log management systems
Using a Log Management System for your WordPress websites provides many benefits that go beyond logging from any one system:
Viewing errors in context
As mentioned previously, one of the major advantages of a log management system is that it aggregates logs to a central location. This can become quite critical when you consider that WordPress does not run in a vacuum – it needs an entire ecosystem of components, tools, and services to run. These include the web server it runs on, the MySQL database it uses to save information, the SMTP server to send emails, and a few other things.
You can begin to analyze the behavior of your environment as a whole, start to connect events and identify patterns. For example, if you experience an issue where WordPress suddenly stops sending emails, the issue might not be with WordPress itself. The SMTP server, or anything else that is required for emails to be sent successfully, could have failed. By having all systems and components report their logs to one central location, it becomes that much easier to troubleshoot such issues without having to go back and forth between all of the different systems.
Other useful features
Log management systems also come with other useful features, such as Loggly’s Surround Search. This tool allows you to view the surrounding events that occurred just before or after critical events across your entire monitored environment. Seeing events in the context of other events can be a great time saver when troubleshooting and can help you identify the root cause of issues that much quicker.
Holistically viewing and searching logs from all your services in a central place is the first step towards analyzing the behavior of your website at scale. For example, when a user enters your website, their ability to log in and access the site may depend on multiple services, such as authentication, localization, and content management.
To troubleshoot any anomalies, you need to be able to track the process from end to end, which means you need to connect multiple events from multiple services. Log management tools that aggregate and allow you to view related events can dramatically shorten issue identification and resolution times.
Fast search across large volumes of events
A key value proposition of log management services is the ability to search and filter large datasets with minimal effort. Log management services use more user-friendly query syntax instead of the complex regular expressions synonymous with traditional tools like awk and grep. You can string simple Boolean keywords and operators, such as AND or OR, and use parentheses for grouping to build very targeted and complex expressions without needing to learn a new query language.
Beyond just searching log files, some log management services, such as Papertrail, also offer the ability to see event messages as they are written. This feature is called Tailing. Tailing a log is very useful both for reproducing an issue, as well as for testing fixes. For example, you can do this on Papertrail with their Live Tail feature.
Scalable and Secure Storage
When there’s an issue or an outage, it’s easy for the size of log files to grow exponentially. If your log management service relies on physical storage or hardware, you can quickly run out of space for new log data and end up losing critical event messages. This is one of the main advantages that log management as a service offers. Cloud-based log management services avoid this issue by allowing log storage capacity to seamlessly scale up as log volume increases. This ability can be a lifesaver when you are facing major system failures.
Since storing data in the cloud isn’t without risks, you’ll need to weigh the ability to scale against your security needs. However, most providers do build log management services with security in mind. They:
- Encrypt the log data using encryption procotols such as AES-256
- Make the data available only through Transport Layer Security (TLS)
Frequently asked questions
Log management and Security Information and Events Management (SIEM) both employ log files to improve security. Here are a few crucial differences:
While SIEM tools and approaches’ primary function is security, Log Management Systems are used more broadly to manage resources, troubleshoot network or application outages, and maintain compliance.
SIEM systems are built to ingest and search limited amounts of selected data. They possess security-related reporting features. An LMS tends to incorporate data from a wider variety of sources – providing a fuller picture at a lower cost and with less maintenance.
Log management is the overall process by which log data is taken from different sources within an organization’s infrastructure. On the other hand, log management software collates log data into one place where it can be stored, then monitored or analyzed.
Log monitoring is the step before log analytics. Here, log data is tracked in real-time. It usually involves the setting up of alerts and notifications, as well as monitoring system performance. The WP Activity Log enables you to monitor log activity in real-time, among other functions.
Log analytics is about the analysis of log data to identify issues, draw connections, and uncover trends.
Here are some general guidelines to help you select the best log management system.
1. While features are important, select a system that you can customize to suit your business needs, team size, compliance obligations, and business policies
2. Since the point of using an LMS is to collate many logs into one, you need to consider which logging formats it supports so you can consolidate as many as possible. Your LMS should be able to collect data from many different sources with different kinds of data
3. Check whether the LMS you are considering has search capabilities and filters
4. When selecting your system, prioritize one with as many automation tools as possible
5. Whatever system you choose, ensure it meets the relevant security and compliance requirements of your industry, sector, and users
6. Consider investing in a cloud-based solution rather than an on-premises one to reduce your security risk and the strain on your network capacity. Offsite records are important for troubleshooting if your entire network crashes, and will provide extra capacity for scaling up as your log file volume increases
7. Invest in whatever training or tutorials you need so all features are utilized
What’s Next?
Check out the full suite of features offered by our WP Activity Log plugin. It offers a configurable and comprehensive activity log that includes third-party events; offers multi-site network support; integrates with many other tools; assists with compliance; and helps you track down precise activities.
