alluxio.job.plan.transform.format.parquet

Class ParquetWriter

java.lang.Object

alluxio.job.plan.transform.format.parquet.ParquetWriter

All Implemented Interfaces:

TableWriter, Closeable, AutoCloseable

public final class ParquetWriter
extends Object
implements TableWriter

A writer for writing ParquetRow.

Method Summary

Modifier and Type	Method and Description
void	close() Closes a writer, which means the table is complete now.
static ParquetWriter	create(TableSchema schema, AlluxioURI uri) Creates a Parquet writer.
static ParquetWriter	create(TableSchema schema, AlluxioURI uri, int rowGroupSize, boolean enableDictionary) Creates a Parquet writer specifying a row group size and whether to have dictionary enabled.
static ParquetWriter	create(TableSchema schema, AlluxioURI uri, int rowGroupSize, boolean enableDictionary, String compressionCodec) Creates a Parquet writer specifying a row group size.
static ParquetWriter	create(TableSchema schema, AlluxioURI uri, PartitionInfo partitionInfo) Creates a parquet writer based on the partitionInfo.
long	getBytes() Note that the bytes written should take compression and encoding into consideration.
int	getRows()
void	write(TableRow row) Writes a row.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

create

public static ParquetWriter create(TableSchema schema,
                                   AlluxioURI uri)
                            throws IOException

Creates a Parquet writer.

Specified by:

create in interface TableWriter

Parameters:

schema - the schema

uri - the URI to the output

Returns:

the writer

Throws:

IOException

create

public static ParquetWriter create(TableSchema schema,
                                   AlluxioURI uri,
                                   @Nullable
                                   PartitionInfo partitionInfo)
                            throws IOException

Creates a parquet writer based on the partitionInfo.

Specified by:

create in interface TableWriter

Parameters:

schema - the schema

uri - the URI to the output

partitionInfo - the partitionInfo (default configuration is used if null)

Returns:

the writer

Throws:

IOException

create

public static ParquetWriter create(TableSchema schema,
                                   AlluxioURI uri,
                                   int rowGroupSize,
                                   boolean enableDictionary)
                            throws IOException

Creates a Parquet writer specifying a row group size and whether to have dictionary enabled.

Parameters:

schema - the schema

uri - the URI to the output

rowGroupSize - the row group size

enableDictionary - whether to enable dictionary

Returns:

the writer

Throws:

IOException

create

public static ParquetWriter create(TableSchema schema,
                                   AlluxioURI uri,
                                   int rowGroupSize,
                                   boolean enableDictionary,
                                   String compressionCodec)
                            throws IOException

Creates a Parquet writer specifying a row group size.

Parameters:

schema - the schema

uri - the URI to the output

rowGroupSize - the row group size

enableDictionary - whether to enable dictionary

compressionCodec - the compression codec name

Returns:

the writer

Throws:

IOException

write

public void write(TableRow row)
           throws IOException

Description copied from interface: TableWriter

Writes a row.

Specified by:

write in interface TableWriter

Parameters:

row - a row

Throws:

IOException - when write fails

close

public void close()
           throws IOException

Description copied from interface: TableWriter

Closes a writer, which means the table is complete now.

Specified by:

close in interface TableWriter

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException - when failing to close the underlying output stream

getRows

public int getRows()

Specified by:

getRows in interface TableWriter

Returns:

the number of rows that have been written

getBytes

public long getBytes()

Description copied from interface: TableWriter

Note that the bytes written should take compression and encoding into consideration. If the writer writes to a file, the bytes written should be an estimate of the actual bytes written to the file.

Specified by:

getBytes in interface TableWriter

Returns:

the number of bytes that have been written