Package org.apache.cassandra.spark.sparksql

Class PartitionSizeIterator

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable, org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.catalyst.InternalRow>
    public class PartitionSizeIterator
extends java.lang.Object
implements org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.catalyst.InternalRow>
    Wrapper iterator around IndexIterator to read all Index.db files and return SparkSQL rows containing all partition keys and the associated on-disk uncompressed and compressed sizes.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()  
      org.apache.spark.sql.catalyst.InternalRow get()  
      boolean next()
      The expected schema is defined in DataLayer.partitionSizeStructType().

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

      • Methods inherited from interface org.apache.spark.sql.connector.read.PartitionReader

        currentMetricsValues

    • Constructor Detail

      • PartitionSizeIterator

        public PartitionSizeIterator​(int partitionId,
                             @NotNull
                             DataLayer dataLayer)

    • Method Detail

      • next

        public boolean next()
             throws java.io.IOException
        The expected schema is defined in DataLayer.partitionSizeStructType(). It consists of the Cassandra partition keys, appended with the columns "uncompressed" and "compressed".
        Specified by:
        next in interface org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.catalyst.InternalRow>
        Throws:
        java.io.IOException

      • get

        public org.apache.spark.sql.catalyst.InternalRow get()
        Specified by:
        get in interface org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.catalyst.InternalRow>

      • close

        public void close()
           throws java.io.IOException
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Throws:
        java.io.IOException