Package org.apache.nutch.fetcher
Class FetchItemQueue
- java.lang.Object
-
- org.apache.nutch.fetcher.FetchItemQueue
-
public class FetchItemQueue extends Object
This class handles FetchItems which come from the same host ID (be it a proto/hostname or proto/IP pair). It also keeps track of requests in progress and elapsed time between requests.
-
-
Constructor Summary
Constructors Constructor Description FetchItemQueue(Configuration conf, int maxThreads, long crawlDelay, long minCrawlDelay)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidaddFetchItem(FetchItem it)voidaddInProgressFetchItem(FetchItem it)voiddump()intemptyQueue()voidfinishFetchItem(FetchItem it, boolean asap)TextgetCookie()FetchItemgetFetchItem()intgetInProgressSize()intgetQueueSize()intincrementExceptionCounter()voidsetCookie(Text cookie)
-
-
-
Constructor Detail
-
FetchItemQueue
public FetchItemQueue(Configuration conf, int maxThreads, long crawlDelay, long minCrawlDelay)
-
-
Method Detail
-
emptyQueue
public int emptyQueue()
-
getQueueSize
public int getQueueSize()
-
getInProgressSize
public int getInProgressSize()
-
incrementExceptionCounter
public int incrementExceptionCounter()
-
finishFetchItem
public void finishFetchItem(FetchItem it, boolean asap)
-
addFetchItem
public void addFetchItem(FetchItem it)
-
addInProgressFetchItem
public void addInProgressFetchItem(FetchItem it)
-
getFetchItem
public FetchItem getFetchItem()
-
setCookie
public void setCookie(Text cookie)
-
getCookie
public Text getCookie()
-
dump
public void dump()
-
-