Flume内置channel,source,sink汇总

翔子

由于经常会使用到Flume的一些channel,source,sink，于是为了方便将这些channel,source,sink汇总出来。

Component Interface	Type Alias	Implementation Class
OAF.Channel	memory	OAF.channel.MemoryChannel
OAF.Channel	jdbc	OAF.channel.jdbc.JdbcChannel
OAF.Channel	file	OAF.channel.file.FileChannel
OAF.Channel	–	OAF.channel.PseudoTxnMemoryChannel
OAF.Channel	–	org.example.MyChannel
OAF.Source	avro	OAF.source.AvroSource
OAF.Source	netcat	OAF.source.NetcatSource
OAF.Source	seq	OAF.source.SequenceGeneratorSource
OAF.Source	exec	OAF.source.ExecSource
OAF.Source	syslogtcp	OAF.source.SyslogTcpSource
OAF.Source	multiport_syslogtcp	OAF.source.MultiportSyslogTCPSource
OAF.Source	syslogudp	OAF.source.SyslogUDPSource
OAF.Source	spooldir	OAF.source.SpoolDirectorySource
OAF.Source	http	OAF.source.http.HTTPSource
OAF.Source	thrift	OAF.source.ThriftSource
OAF.Source	jms	OAF.source.jms.JMSSource
OAF.Source	–	OAF.source.avroLegacy.AvroLegacySource
OAF.Source	–	OAF.source.thriftLegacy.ThriftLegacySource
OAF.Source	–	org.example.MySource
OAF.Sink	null	OAF.sink.NullSink
OAF.Sink	logger	OAF.sink.LoggerSink
OAF.Sink	avro	OAF.sink.AvroSink
OAF.Sink	hdfs	OAF.sink.hdfs.HDFSEventSink
OAF.Sink	hbase	OAF.sink.hbase.HBaseSink
OAF.Sink	asynchbase	OAF.sink.hbase.AsyncHBaseSink
OAF.Sink	elasticsearch	OAF.sink.elasticsearch.ElasticSearchSink
OAF.Sink	file_roll	OAF.sink.RollingFileSink
OAF.Sink	irc	OAF.sink.irc.IRCSink
OAF.Sink	thrift	OAF.sink.ThriftSink
OAF.Sink	–	org.example.MySink
OAF.ChannelSelector	replicating	OAF.channel.ReplicatingChannelSelector
OAF.ChannelSelector	multiplexing	OAF.channel.MultiplexingChannelSelector
OAF.ChannelSelector	–	org.example.MyChannelSelector
OAF.SinkProcessor	default	OAF.sink.DefaultSinkProcessor
OAF.SinkProcessor	failover	OAF.sink.FailoverSinkProcessor
OAF.SinkProcessor	load_balance	OAF.sink.LoadBalancingSinkProcessor
OAF.SinkProcessor	–
OAF.interceptor.Interceptor	timestamp	OAF.interceptor.TimestampInterceptor$Builder
OAF.interceptor.Interceptor	host	OAF.interceptor.HostInterceptor$Builder
OAF.interceptor.Interceptor	static	OAF.interceptor.StaticInterceptor$Builder
OAF.interceptor.Interceptor	regex_filter	OAF.interceptor.RegexFilteringInterceptor$Builder
OAF.interceptor.Interceptor	regex_extractor	OAF.interceptor.RegexFilteringInterceptor$Builder
OAF.channel.file.encryption. KeyProvider$Builder	jceksfile	OAF.channel.file.encryption.JCEFileKeyProvider
OAF.channel.file.encryption. KeyProvider$Builder	–	org.example.MyKeyProvider
OAF.channel.file.encryption.CipherProvider	aesctrnopadding	OAF.channel.file.encryption.AESCTRNoPaddingProvider
OAF.channel.file.encryption.CipherProvider	–	org.example.MyCipherProvider
OAF.serialization.EventSerializer$Builder	text	OAF.serialization.BodyTextEventSerializer$Builder
OAF.serialization.EventSerializer$Builder	avro_event	OAF.serialization.FlumeEventAvroEventSerializer$Builder
OAF.serialization.EventSerializer$Builder	–	org.example.MyEventSerializer$Builder

由于表格太大了，所以这里将org.apache.flume.替换为OAF.，读者在阅读的时候，直接将OAF.理解成org.apache.flume.即可！

Sinks accumulator agentBEChain agentBESink agentBestEffortSink agentDFOChain agentDFOSink agentE2EChain agentE2ESink agentFailoverSink agentSink attr2hbase autoBEChain autoDFOChain autoE2EChain avroSink collectorSink console counter counterHistory customdfs dfs escapedCustomDfs escapedFormatDfs fail failChain flightbookinghbase flightsearchhbase formatDfs ganglia hbase irc logicalSink multigrep multigrepspec null regexhisto regexhistospec rpcSink seqfile syslogTcp text thriftSink

Sources asciisynth autoCollectorSource avroSource collectorSource console exec execPeriodic execStream fail irc log4jfile logicalSource multitail nonlsynth null report rpcSource scribe seqfile stdin synth synthrndsize syslogTcp syslogTcp1 syslogUdp tail tailDir text thriftSource tpriosource twitter

Decorators AccessIpFormatFilter AmendAccessLogNew AmendNginxErrorLog ApacheFilter CatalinaFormat CatalinaLogExceptionFilter2 CatalinaLogExceptionFilter3 CatalinaLogFilter2 CatalinaLogFilter3 FieldEqValueFilter Filter FormatAccessLog FormatDate LogExceptionFilter LogFilter PhpFilter ackChecker ackInjector ackedWriteAhead amendAccessLog batch benchinject benchreport bloomCheck bloomGen choke delay digest diskFailover exDate flakeyAppend format gunzip gzip inmem insertBefore insistentAppend insistentOpen intervalDroppyAppend intervalFlakeyAppend intervalSampler lazyOpen mask mult nullDeco probSampler regex regexAll reservoirSampler select split splitCompare stubbornAppend unbatch value