Class TCLogParser
java.lang.Object
org.apache.jmeter.protocol.http.util.accesslog.TCLogParser
- All Implemented Interfaces:
LogParser
- Direct Known Subclasses:
SharedTCLogParser
Description:
Currently the parser only handles GET/POST requests. It's easy enough to add support for other request methods by changing checkMethod. The is a complete rewrite of a tool I wrote for myself earlier. The older algorithm was basic and did not provide the same level of flexibility I want, so I wrote a new one using a totally new algorithm. This implementation reads one line at a time using BufferedReader. When it gets to the end of the file and the sampler needs to get more requests, the parser will re-initialize the BufferedReader. The implementation uses StringTokenizer to create tokens.
Currently the parser only handles GET/POST requests. It's easy enough to add support for other request methods by changing checkMethod. The is a complete rewrite of a tool I wrote for myself earlier. The older algorithm was basic and did not provide the same level of flexibility I want, so I wrote a new one using a totally new algorithm. This implementation reads one line at a time using BufferedReader. When it gets to the end of the file and the sampler needs to get more requests, the parser will re-initialize the BufferedReader. The implementation uses StringTokenizer to create tokens.
The parse algorithm is the following:
- cleans the entry by looking for backslash "\"
- looks to see if GET or POST is in the line
- tokenizes using quotes "
- finds the token with the request method
- gets the string of the token and tokenizes it using space
- finds the first token beginning with slash character
- tokenizes the string using question mark "?"
- get the path from the first token
- returns the second token and checks it for parameters
- tokenizes the string using ampersand "&"
- parses each token to name/value pairs
Extending this class is fairly simple. Most access logs use the same format starting from the request method. Therefore, changing the implementation of cleanURL(string) method should be sufficient to support new log formats. Tomcat uses common log format, so any webserver that uses the format should work with this parser. Servers that are known to use non standard formats are IIS and Netscape.
-
Field Summary
FieldsModifier and TypeFieldDescriptionprotected booleanby default, we probably should decode the parameter valuesprotected Stringprotected FilterHandles to supporting classesstatic final Stringstatic final Stringprotected static final org.slf4j.Loggerstatic final Stringprotected BufferedReaderprotected Stringprotected members *protected Fileprotected StringThe path to the access log fileprotected boolean -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionbooleancheckMethod(String text) The method checks forPOST,GETandHEADmethods currently.booleancheckParamFormat(String text) Checks the string to see if it contains "&" and "=".booleanChecks the string to make sure it has/path/file?name=valueformat.The method cleans the URL using the following algorithm.voidclose()close the any streams or readers.voidconvertStringToJMRequest(String text, org.apache.jmeter.testelement.TestElement el) Convert a single line into XMLNVPair[]convertStringtoNVPair(String stringparams) Parse the string parameters into NVPair[] array.booleandecode the parameter values is to true by defaultprotected intparse(BufferedReader breader, org.apache.jmeter.testelement.TestElement el, int parseCount) The method is responsible for reading each line, and breaking out of the while loop if a set number of lines is given.
Note: empty lines will not be countedintparse(org.apache.jmeter.testelement.TestElement el, int parseCount) parse the entire file.intparseAndConfigure(int count, org.apache.jmeter.testelement.TestElement el) parse a set number of lines from the access log.protected intparseLine calls the other parse methods to parse the given text.protected NVPairparseOneParameter(String parameter) Method expects name and value to be separated by an equal sign "=".parseParameters(String parameters) Method uses StringTokenizer to convert the string into single pairs.voidsetDecodeParameterValues(boolean decodeparams) by default decode is set to true.voidUse the filter to include/exclude files in the access logs.voidsetSourceFile(String source) Sets the source file.voidsetUseParsedFile(boolean file) Calls this method to set whether or not to use the path in the log.Tokenize the URL into two tokens.Parses the line using java.util.StringTokenizer.
-
Field Details
-
log
protected static final org.slf4j.Logger log -
GET
- See Also:
-
POST
- See Also:
-
HEAD
- See Also:
-
RMETHOD
protected members * -
URL_PATH
The path to the access log file -
useFILE
protected boolean useFILE -
SOURCE
-
FILENAME
-
READER
-
FILTER
Handles to supporting classes -
decode
protected boolean decodeby default, we probably should decode the parameter values
-
-
Constructor Details
-
TCLogParser
public TCLogParser() -
TCLogParser
- Parameters:
source- name of the source file
-
-
Method Details
-
setDecodeParameterValues
public void setDecodeParameterValues(boolean decodeparams) by default decode is set to true. if the parameters shouldn't be decoded, call the method with false- Parameters:
decodeparams- flag whether parameters should be decoded
-
decodeParameterValue
public boolean decodeParameterValue()decode the parameter values is to true by default- Returns:
trueif parameter values should be decoded,falseotherwise
-
setUseParsedFile
public void setUseParsedFile(boolean file) Calls this method to set whether or not to use the path in the log. We may want to provide the ability to filter the log file later on. By default, the parser uses the file in the log.- Parameters:
file- flag whether to use the path from the log
-
setFilter
Use the filter to include/exclude files in the access logs. This is provided as a convenience and reduce the need to spend hours cleaning up log files. -
setSourceFile
Sets the source file.- Specified by:
setSourceFilein interfaceLogParser- Parameters:
source- name of the source file
-
parse
public int parse(org.apache.jmeter.testelement.TestElement el, int parseCount) parse the entire file.- Parameters:
el- TestElement to read the lines intoparseCount- number of max lines to read- Returns:
- number of read lines, or
-1if an error occurred while reading
-
parseAndConfigure
public int parseAndConfigure(int count, org.apache.jmeter.testelement.TestElement el) parse a set number of lines from the access log. Keep in mind the number of lines parsed will depend on the filter and number of lines in the log. The method returns the actual number of lines parsed.- Specified by:
parseAndConfigurein interfaceLogParser- Parameters:
count- number of lines to readel-TestElementto read lines into- Returns:
- lines parsed
-
parse
protected int parse(BufferedReader breader, org.apache.jmeter.testelement.TestElement el, int parseCount) The method is responsible for reading each line, and breaking out of the while loop if a set number of lines is given.
Note: empty lines will not be counted- Parameters:
breader-BufferedReaderto read lines fromel-TestElementto read lines intoparseCount- number of lines to read- Returns:
- number of lines parsed
-
parseLine
parseLine calls the other parse methods to parse the given text.- Parameters:
line- single line to be parsedel-TestElementin which the line will be added- Returns:
- number of lines parsed (zero or one, actually)
-
cleanURL
The method cleans the URL using the following algorithm.- check for double quotes
- check the request method
- tokenize using double quotes
- find first token containing request method
- tokenize string using space
- find first token that begins with "/"
127.0.0.1 - - [08/Jan/2003:07:03:54 -0500] "GET /addrbook/ HTTP/1.1" 200 1981
would result in the extracted url
/addrbook/- Parameters:
entry- line from which the url is to be extracted- Returns:
- cleaned url
-
checkMethod
The method checks forPOST,GETandHEADmethods currently. The other methods aren't supported yet.- Parameters:
text- text to be checked for HTTP method- Returns:
trueif method is supported,falseotherwise
-
stripFile
Tokenize the URL into two tokens. If the URL has more than one "?", the parse may fail. Only the first two tokens are used. The first token is automatically parsed and set at TCLogParser#URL_PATH.- Parameters:
url- url which should be stripped from parametersel-TestElementto parse url into- Returns:
- String presenting the parameters, or
nullwhen none where found
-
checkURL
Checks the string to make sure it has/path/file?name=valueformat. If the string doesn't contains a "?", it will returnfalse.- Parameters:
url- url to check for parameters- Returns:
trueif url contains a?,falseotherwise
-
checkParamFormat
Checks the string to see if it contains "&" and "=". If it does, returntrue, so that it can be parsed.- Parameters:
text- text to be checked for&and=- Returns:
trueiftextcontains both&and=,falseotherwise
-
convertStringToJMRequest
Convert a single line into XML- Parameters:
text- to be convertedel-HTTPSamplerBasewhich consumes thetext
-
convertStringtoNVPair
Parse the string parameters into NVPair[] array. Once they are parsed, it is returned. The method uses parseOneParameter(string) to convert each pair.- Parameters:
stringparams- String with parameters to be parsed- Returns:
- array of
NVPairs
-
parseOneParameter
Method expects name and value to be separated by an equal sign "=". The method uses StringTokenizer to make a NVPair object. If there happens to be more than one "=" sign, the others are ignored. The chance of a string containing more than one is unlikely and would not conform to HTTP spec. I should double check the protocol spec to make sure this is accurate.- Parameters:
parameter- to be parsed- Returns:
NVPairwith the parsed name and value of the parameter
-
parseParameters
Method uses StringTokenizer to convert the string into single pairs. The string should conform to HTTP protocol spec, which means the name/value pairs are separated by the ampersand symbol "&". Someone could write the querystrings by hand, but that would be round about and go against the purpose of this utility.- Parameters:
parameters- string to be parsed- Returns:
- List of name/value pairs
-
tokenize
Parses the line using java.util.StringTokenizer.- Parameters:
line- line to be parseddelim- delimiter- Returns:
- StringTokenizer constructed with
lineanddelim
-
close
public void close()Description copied from interface:LogParserclose the any streams or readers.
-