Groundwork Groundwork

Practice: a log line parser

A server writes a line every time something happens, and by Friday there are forty thousand of them. Each line looks like this:

WARN|14:03|Disk usage at 91 percent

Three fields separated by |: the level, the time, and the message. The message is never empty and never contains a |. You are going to write the methods that pull a line apart, and then one that finds the lines worth reading.

Everything here uses Chapters 1 to 8: arrays, ArrayList, loops, and now split, String.join, toUpperCase, strip, replace and Character.isLetter.

Real logs are untidy. Some lines come in with the level in lowercase and stray spaces around the message, so a couple of these steps have to cope with that rather than assume it away.

Each step names the method exactly: its return type, its name and its parameters. Check calls it with the cases under "What it should do," and step 8's cases write each list of lines in place, as new ArrayList<>(List.of(...)).

Steps

Step 1. Write String levelOf(String line), returning the level, so WARN from the line above. Split on the separator rather than counting characters, remembering that | is special in a pattern.

What it should do:

  • levelOf("WARN|14:03|Disk usage at 91 percent") -> "WARN"
  • levelOf("INFO|09:15|Backup finished") -> "INFO"
  • levelOf("ERROR|00:01|Down") -> "ERROR"
Show a solution
String levelOf(String line) {
    return line.split("\\|")[0];
}

void main() {
    IO.println(levelOf("WARN|14:03|Disk usage at 91 percent"));
}

Split on the separator the format actually uses, and the level is the first piece, index 0. | is special in a pattern, so it is written "\\|". Counting characters would work for WARN and INFO and break on ERROR.

Step 2. Write String timeOf(String line), returning the time field.

What it should do:

  • timeOf("WARN|14:03|Disk usage at 91 percent") -> "14:03"
  • timeOf("INFO|09:15|Backup finished") -> "09:15"
  • timeOf("ERROR|00:01|Down") -> "00:01"
Show a solution
String timeOf(String line) {
    return line.split("\\|")[1];
}

void main() {
    IO.println(timeOf("WARN|14:03|Disk usage at 91 percent"));
}

The time is the second field, so it is at index 1. Counting from 0 is the whole of this step.

Step 3. Write String messageOf(String line), returning the message field.

What it should do:

  • messageOf("WARN|14:03|Disk usage at 91 percent") -> "Disk usage at 91 percent"
  • messageOf("INFO|09:15|Backup finished") -> "Backup finished"
  • messageOf("ERROR|00:01|Down") -> "Down"
Show a solution
String messageOf(String line) {
    return line.split("\\|")[2];
}

void main() {
    IO.println(messageOf("WARN|14:03|Disk usage at 91 percent"));
}

The message is the third and last field, at index 2.

Step 4. Write boolean isWarning(String line), returning true for a warning and false otherwise. It must say true for warn in lowercase too, because half the services write it that way.

What it should do:

  • isWarning("WARN|14:03|Disk usage at 91 percent") -> true
  • isWarning("INFO|09:15|Backup finished") -> false
  • isWarning("warn|08:00| Fan speed low ") -> true
  • isWarning("ERROR|00:01|Down") -> false
  • isWarning("INFO|10:00|Warning cleared") -> false
Show a solution
boolean isWarning(String line) {
    String level = line.split("\\|")[0];
    return level.toUpperCase().equals("WARN");
}

void main() {
    IO.println(isWarning("WARN|14:03|Disk usage at 91 percent"));
    IO.println(isWarning("warn|08:00|  Fan speed low  "));
    IO.println(isWarning("INFO|09:15|Backup finished"));
}

Put the level in one case before comparing, and compare text with .equals. Without toUpperCase(), half the services are never warned about.

Step 5. Write String tidy(String line), returning the line rebuilt with the level in uppercase and the message stripped of leading and trailing spaces, still separated by |. Use String.join to put it back together.

What it should do:

  • tidy("warn|08:00| Fan speed low ") -> "WARN|08:00|Fan speed low"
  • tidy("WARN|14:03|Disk usage at 91 percent") -> "WARN|14:03|Disk usage at 91 percent"
  • tidy("INFO|09:15|Backup finished") -> "INFO|09:15|Backup finished"
Show a solution
String tidy(String line) {
    String[] fields = line.split("\\|");
    fields[0] = fields[0].toUpperCase();
    fields[2] = fields[2].strip();
    return String.join("|", fields);
}

void main() {
    IO.println(tidy("warn|08:00|  Fan speed low  "));
}

An array's items can be replaced, so the level and the message are swapped for their tidied versions, and String.join puts the separator back between the pieces. String.join takes plain text, not a pattern, so its | needs no backslash. A line that was already tidy comes back unchanged.

Step 6. Write ArrayList<String> wordsIn(String message), returning a list of the message's words that are made only of letters, so Disk usage at 91 percent loses the 91. The message may have stray spaces, like the untidy line's.

What it should do:

  • wordsIn("Disk usage at 91 percent") -> new ArrayList<>(List.of("Disk", "usage", "at", "percent"))
  • wordsIn("Backup finished") -> new ArrayList<>(List.of("Backup", "finished"))
  • wordsIn("91 42") -> new ArrayList<>()
  • wordsIn(" Fan speed low ") -> new ArrayList<>(List.of("Fan", "speed", "low"))
  • wordsIn("Disk sda1 at 91 percent") -> new ArrayList<>(List.of("Disk", "at", "percent"))
Show a solution
ArrayList<String> wordsIn(String message) {
    ArrayList<String> words = new ArrayList<>();
    for (String word : message.strip().split("\\s+")) {
        boolean allLetters = true;
        for (int i = 0; i < word.length(); i++) {
            if (!Character.isLetter(word.charAt(i))) {
                allLetters = false;
            }
        }
        if (allLetters) {
            words.add(word);
        }
    }
    return words;
}

void main() {
    IO.println(wordsIn("Disk usage at 91 percent"));
}

strip().split("\\s+") gives the words however the spaces fall. A word is kept only if every one of its characters is a letter: the flag starts true for each word, and any character that is not a letter turns it false, which is how 91 and sda1 drop out.

Step 7. Write String redact(String line, String word), returning the line with every appearance of that word replaced by ****.

What it should do:

  • redact("WARN|14:03|Disk usage at 91 percent", "91") -> "WARN|14:03|Disk usage at **** percent"
  • redact("a b a", "a") -> "**** b ****"
  • redact("INFO|09:15|Backup finished", "nothing") -> "INFO|09:15|Backup finished"
Show a solution
String redact(String line, String word) {
    return line.replace(word, "****");
}

void main() {
    IO.println(redact("WARN|14:03|Disk usage at 91 percent", "91"));
}

replace changes every appearance, not just the first, and gives back the line unchanged when the word is not there.

Step 8. Write ArrayList<String> warningMessages(ArrayList<String> lines), returning just the messages of the warnings, each stripped of the spaces around it—not the whole lines, and not the other levels.

What it should do:

  • warningMessages(new ArrayList<>(List.of("WARN|14:03|Disk usage at 91 percent", "INFO|09:15|Backup finished", "warn|08:00| Fan speed low "))) -> new ArrayList<>(List.of("Disk usage at 91 percent", "Fan speed low"))
  • warningMessages(new ArrayList<>(List.of("INFO|09:15|Backup finished"))) -> new ArrayList<>()
  • warningMessages(new ArrayList<>()) -> new ArrayList<>()
  • warningMessages(new ArrayList<>(List.of("INFO|10:00|Warning cleared", "warn|08:00| Fan speed low "))) -> new ArrayList<>(List.of("Fan speed low"))
Show a solution
ArrayList<String> warningMessages(ArrayList<String> lines) {
    ArrayList<String> messages = new ArrayList<>();
    for (String line : lines) {
        String[] fields = line.split("\\|");
        if (fields[0].toUpperCase().equals("WARN")) {
            messages.add(fields[2].strip());
        }
    }
    return messages;
}

void main() {
    ArrayList<String> lines = new ArrayList<>();
    lines.add("WARN|14:03|Disk usage at 91 percent");
    lines.add("INFO|09:15|Backup finished");
    lines.add("warn|08:00|  Fan speed low  ");
    IO.println(warningMessages(lines));
}

The messages, not the lines: each warning's third field, stripped. The lowercase line has to be caught and its message tidied, which is why the level is uppercased before the comparison.