Trino S3 via hive-metastore Integration

This blog on Trino S3 initially appeared on Medium. It was republished with the author’s credit and consent.

In this blog, I’ll go over how to use S3 storage on a Pure Storage® FlashBlade® with Trino, the fast distributed SQL query engine for big data.

I deploy Trino using the hive chart and provide a values.yaml file with the following configuration:

<span style="font-weight: 400;">additionalCatalogs:</span>

<span style="font-weight: 400;"> </span><span style="font-weight: 400;">lakehouse:</span> <span style="font-weight: 400;">|-</span>

<span style="font-weight: 400;">   connector.name=hive</span>

<span style="font-weight: 400;">   hive.metastore.uri=thrift://hivems-hive-metastore.analytics.svc.local:9083   </span>

<span style="font-weight: 400;">   hive.s3.aws-access-key=PSFB....BJEIA</span>

<span style="font-weight: 400;">   hive.s3.aws-secret-key=A121....eJOEN</span>

<span style="font-weight: 400;">   hive.s3.endpoint=192.168.2.2</span>

<span style="font-weight: 400;">   hive.s3.ssl.enabled=false</span>

additionalCatalogs:

lakehouse: |-

connector.name=hive

hive.metastore.uri=thrift://hivems-hive-metastore.analytics.svc.local:9083

hive.s3.aws-access-key=PSFB....BJEIA

hive.s3.aws-secret-key=A121....eJOEN

hive.s3.endpoint=192.168.2.2

hive.s3.ssl.enabled=false

This is pointing to my hive-metastore server. See this blog post for more information on setting that up. I then edit the Trino service to switch from ClusterIP to NodePort to facilitate external access.

As usual, I use the helm install command:

<span style="font-weight: 400;">helm -n analytics install trino trino/trino  -f values.yaml</span>

1	<span style="font-weight: 400;">helm -n analytics install trino trino/trino -f values.yaml</span>

On a Linux client with the trino-cli installed, I use the following command to connect to my in Kubernetes running instance, and list the current catalogs available:

<span style="font-weight: 400;">$ ./trino-cli</span><span style="font-weight: 400;">-422</span><span style="font-weight: 400;">-executable.jar </span><span style="font-weight: 400;">--server http://10.225.114.50:32140</span>

<span style="font-weight: 400;">trino&gt; </span><span style="font-weight: 400;">show</span><span style="font-weight: 400;"> catalogs;</span>

<span style="font-weight: 400;"> Catalog </span>

<span style="font-weight: 400;">-----------</span>

<span style="font-weight: 400;">hive</span>

<span style="font-weight: 400;">system</span><span style="font-weight: 400;">   </span>

<span style="font-weight: 400;">tpcds    </span>

<span style="font-weight: 400;">tpch     </span>

<span style="font-weight: 400;">(</span><span style="font-weight: 400;">4</span> <span style="font-weight: 400;">rows</span><span style="font-weight: 400;">)</span>

<span style="font-weight: 400;">Query </span><span style="font-weight: 400;">20230810</span><span style="font-weight: 400;">_122520_00000_gb9kf, FINISHED, </span><span style="font-weight: 400;">2</span><span style="font-weight: 400;"> nodes</span>

<span style="font-weight: 400;">Splits: </span><span style="font-weight: 400;">68</span><span style="font-weight: 400;"> total, </span><span style="font-weight: 400;">68</span><span style="font-weight: 400;"> done (</span><span style="font-weight: 400;">100.00</span><span style="font-weight: 400;">%)</span>

<span style="font-weight: 400;">1.15</span><span style="font-weight: 400;"> [</span><span style="font-weight: 400;">0</span> <span style="font-weight: 400;">rows</span><span style="font-weight: 400;">, </span><span style="font-weight: 400;">0</span><span style="font-weight: 400;">B] [</span><span style="font-weight: 400;">0</span> <span style="font-weight: 400;">rows</span><span style="font-weight: 400;">/s, </span><span style="font-weight: 400;">0</span><span style="font-weight: 400;">B/s]</span>

<span style="font-weight: 400;">trino&gt;</span>

$ ./trino-cli-422-executable.jar --server http://10.225.114.50:32140

trino> show catalogs;

Catalog

-----------

hive

system

tpcds

tpch

(4 rows)

Query 20230810_122520_00000_gb9kf, FINISHED, 2 nodes

Splits: 68 total, 68 done (100.00%)

1.15 [0 rows, 0B] [0 rows/s, 0B/s]

trino>

I can then select my hive source and check the available tables:

<span style="font-weight: 400;">trino&gt; use hive.default;</span>

<span style="font-weight: 400;">USE</span>

<span style="font-weight: 400;">trino:</span><span style="font-weight: 400;">default</span><span style="font-weight: 400;">&gt;  </span><span style="font-weight: 400;">show</span><span style="font-weight: 400;"> tables;</span>

<span style="font-weight: 400;">    </span><span style="font-weight: 400;">Table</span><span style="font-weight: 400;">    </span>

<span style="font-weight: 400;">---------------</span>

<span style="font-weight: 400;">nyctaxi_table</span>

<span style="font-weight: 400;">(</span><span style="font-weight: 400;">1</span> <span style="font-weight: 400;">row</span><span style="font-weight: 400;">)</span>

<span style="font-weight: 400;">Query </span><span style="font-weight: 400;">20230810</span><span style="font-weight: 400;">_140256_00004_uapkp, FINISHED, </span><span style="font-weight: 400;">3</span><span style="font-weight: 400;"> nodes</span>

<span style="font-weight: 400;">Splits: </span><span style="font-weight: 400;">68</span><span style="font-weight: 400;"> total, </span><span style="font-weight: 400;">68</span><span style="font-weight: 400;"> done (</span><span style="font-weight: 400;">100.00</span><span style="font-weight: 400;">%)</span>

<span style="font-weight: 400;">0.45</span><span style="font-weight: 400;"> [</span><span style="font-weight: 400;">1</span> <span style="font-weight: 400;">rows</span><span style="font-weight: 400;">, </span><span style="font-weight: 400;">30</span><span style="font-weight: 400;">B] [</span><span style="font-weight: 400;">2</span> <span style="font-weight: 400;">rows</span><span style="font-weight: 400;">/s, </span><span style="font-weight: 400;">67</span><span style="font-weight: 400;">B/s]</span>

<span style="font-weight: 400;">trino:</span><span style="font-weight: 400;">default</span><span style="font-weight: 400;">&gt;</span>

trino> use hive.default;

USE

Table

---------------

nyctaxi_table

(1 row)

Query 20230810_140256_00004_uapkp, FINISHED, 3 nodes

0.45 [1 rows, 30B] [2 rows/s, 67B/s]

trino:default>

Note that to see the available schemas, you can use:

trino> show schemas from hive;

Schema

——————–

default

information_schema

(2 rows)

Query 20230810_152954_00003_a45jd, FINISHED, 3 nodes

Splits: 68 total, 68 done (100.00%)

0.61 [2 rows, 35B] [3 rows/s, 57B/s]

To see the table schema, you can use:

<span style="font-weight: 400;">trino:</span><span style="font-weight: 400;">default</span><span style="font-weight: 400;">&gt; </span><span style="font-weight: 400;">describe</span><span style="font-weight: 400;"> nyctaxi;</span>

<span style="font-weight: 400;">       </span><span style="font-weight: 400;">Column</span><span style="font-weight: 400;">         |     Type     | Extra | Comment</span>

<span style="font-weight: 400;">-----------------------+--------------+-------+---------</span>

<span style="font-weight: 400;">vendorid              | </span><span style="font-weight: 400;">bigint</span><span style="font-weight: 400;">       |       |        </span>

<span style="font-weight: 400;">tpep_pickup_datetime  | </span><span style="font-weight: 400;">timestamp</span><span style="font-weight: 400;">(</span><span style="font-weight: 400;">3</span><span style="font-weight: 400;">) |       |        </span>

<span style="font-weight: 400;">tpep_dropoff_datetime | </span><span style="font-weight: 400;">timestamp</span><span style="font-weight: 400;">(</span><span style="font-weight: 400;">3</span><span style="font-weight: 400;">) |       |        </span>

<span style="font-weight: 400;">passenger_count       | </span><span style="font-weight: 400;">double</span><span style="font-weight: 400;">       |       |        </span>

<span style="font-weight: 400;">trip_distance         | </span><span style="font-weight: 400;">double</span><span style="font-weight: 400;">       |       |        </span>

<span style="font-weight: 400;">ratecodeid            | </span><span style="font-weight: 400;">double</span><span style="font-weight: 400;">       |       |        </span>

<span style="font-weight: 400;">store_and_fwd_flag    | </span><span style="font-weight: 400;">varchar</span><span style="font-weight: 400;">      |       |        </span>

<span style="font-weight: 400;">pulocationid          | </span><span style="font-weight: 400;">bigint</span><span style="font-weight: 400;">       |       |        </span>

<span style="font-weight: 400;">dolocationid          | </span><span style="font-weight: 400;">bigint</span><span style="font-weight: 400;">       |       |        </span>

<span style="font-weight: 400;">payment_type          | </span><span style="font-weight: 400;">bigint</span><span style="font-weight: 400;">       |       |        </span>

<span style="font-weight: 400;">fare_amount           | </span><span style="font-weight: 400;">double</span><span style="font-weight: 400;">       |       |        </span>

<span style="font-weight: 400;">extra                 | </span><span style="font-weight: 400;">double</span><span style="font-weight: 400;">       |       |        </span>

<span style="font-weight: 400;">mta_tax               | </span><span style="font-weight: 400;">double</span><span style="font-weight: 400;">       |       |        </span>

<span style="font-weight: 400;">tip_amount            | </span><span style="font-weight: 400;">double</span><span style="font-weight: 400;">       |       |        </span>

<span style="font-weight: 400;">tolls_amount          | </span><span style="font-weight: 400;">double</span><span style="font-weight: 400;">       |       |        </span>

<span style="font-weight: 400;">improvement_surcharge | </span><span style="font-weight: 400;">double</span><span style="font-weight: 400;">       |       |        </span>

<span style="font-weight: 400;">total_amount          | </span><span style="font-weight: 400;">double</span><span style="font-weight: 400;">       |       |        </span>

<span style="font-weight: 400;">(</span><span style="font-weight: 400;">17</span> <span style="font-weight: 400;">rows</span><span style="font-weight: 400;">)</span>

<span style="font-weight: 400;">Query </span><span style="font-weight: 400;">20230810</span><span style="font-weight: 400;">_163505_00104_a45jd, FINISHED, </span><span style="font-weight: 400;">3</span><span style="font-weight: 400;"> nodes</span>

<span style="font-weight: 400;">Splits: </span><span style="font-weight: 400;">68</span><span style="font-weight: 400;"> total, </span><span style="font-weight: 400;">68</span><span style="font-weight: 400;"> done (</span><span style="font-weight: 400;">100.00</span><span style="font-weight: 400;">%)</span>

<span style="font-weight: 400;">0.28</span><span style="font-weight: 400;"> [</span><span style="font-weight: 400;">17</span> <span style="font-weight: 400;">rows</span><span style="font-weight: 400;">, </span><span style="font-weight: 400;">1.05</span><span style="font-weight: 400;">KB] [</span><span style="font-weight: 400;">60</span> <span style="font-weight: 400;">rows</span><span style="font-weight: 400;">/s, </span><span style="font-weight: 400;">3.72</span><span style="font-weight: 400;">KB/s]</span>

Column | Type | Extra | Comment

-----------------------+--------------+-------+---------

vendorid | bigint | |

tpep_pickup_datetime | timestamp(3) | |

tpep_dropoff_datetime | timestamp(3) | |

passenger_count | double | |

trip_distance | double | |

ratecodeid | double | |

store_and_fwd_flag | varchar | |

pulocationid | bigint | |

dolocationid | bigint | |

payment_type | bigint | |

fare_amount | double | |

extra | double | |

mta_tax | double | |

tip_amount | double | |

tolls_amount | double | |

improvement_surcharge | double | |

total_amount | double | |

(17 rows)

Query 20230810_163505_00104_a45jd, FINISHED, 3 nodes

0.28 [17 rows, 1.05KB] [60 rows/s, 3.72KB/s]

I can now run various queries on the data set. Please note this is from a very limited lab Kubernetes cluster with low resources and network connectivity, so performance was not the aim:

<span style="font-weight: 400;">trino:</span><span style="font-weight: 400;">default</span><span style="font-weight: 400;">&gt; </span><span style="font-weight: 400;">select</span> <span style="font-weight: 400;">count</span><span style="font-weight: 400;">(*) </span><span style="font-weight: 400;">from</span><span style="font-weight: 400;"> nyctaxi;</span>

<span style="font-weight: 400;">  _col0  </span>

<span style="font-weight: 400;">-----------</span>

<span style="font-weight: 400;">899294289</span>

<span style="font-weight: 400;">(</span><span style="font-weight: 400;">1</span> <span style="font-weight: 400;">row</span><span style="font-weight: 400;">)</span>

<span style="font-weight: 400;">Query </span><span style="font-weight: 400;">20230815</span><span style="font-weight: 400;">_132013_00029_4suvq, FINISHED, </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> node</span>

<span style="font-weight: 400;">Splits: </span><span style="font-weight: 400;">343</span><span style="font-weight: 400;"> total, </span><span style="font-weight: 400;">343</span><span style="font-weight: 400;"> done (</span><span style="font-weight: 400;">100.00</span><span style="font-weight: 400;">%)</span>

<span style="font-weight: 400;">1.04</span><span style="font-weight: 400;"> [</span><span style="font-weight: 400;">899</span><span style="font-weight: 400;">M </span><span style="font-weight: 400;">rows</span><span style="font-weight: 400;">, </span><span style="font-weight: 400;">3.56</span><span style="font-weight: 400;">MB] [</span><span style="font-weight: 400;">865</span><span style="font-weight: 400;">M </span><span style="font-weight: 400;">rows</span><span style="font-weight: 400;">/s, </span><span style="font-weight: 400;">3.43</span><span style="font-weight: 400;">MB/s]</span>

<span style="font-weight: 400;">trino:</span><span style="font-weight: 400;">default</span><span style="font-weight: 400;">&gt; </span><span style="font-weight: 400;">select</span><span style="font-weight: 400;"> * </span><span style="font-weight: 400;">from</span><span style="font-weight: 400;"> nyctaxi </span><span style="font-weight: 400;">WHERE</span><span style="font-weight: 400;"> tolls_amount &gt; </span><span style="font-weight: 400;">100</span> <span style="font-weight: 400;">order</span> <span style="font-weight: 400;">by</span><span style="font-weight: 400;"> tolls_amount </span><span style="font-weight: 400;">desc</span><span style="font-weight: 400;"> limit </span><span style="font-weight: 400;">10</span><span style="font-weight: 400;">;</span>

<span style="font-weight: 400;">vendorid |  tpep_pickup_datetime   |  tpep_dropoff_datetime  | passenger_count | trip_distance | ratecodeid | store_and_fwd_flag | pulocationid | dolocationid | payment_type | fare_amount | extra | mta</span>

<span style="font-weight: 400;">----------+-------------------------+-------------------------+-----------------+---------------+------------+--------------------+--------------+--------------+--------------+-------------+-------+----</span>

<span style="font-weight: 400;">       </span><span style="font-weight: 400;">2</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2017-07-03</span> <span style="font-weight: 400;">17</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">15</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">15.000</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2017-07-03</span> <span style="font-weight: 400;">17</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">35</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">57.000</span><span style="font-weight: 400;"> |               </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">9.74</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> | N                  |          </span><span style="font-weight: 400;">138</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">107</span><span style="font-weight: 400;"> |            </span><span style="font-weight: 400;">2</span><span style="font-weight: 400;"> |        </span><span style="font-weight: 400;">27.5</span><span style="font-weight: 400;"> |   </span><span style="font-weight: 400;">1.0</span><span style="font-weight: 400;"> |   </span>

<span style="font-weight: 400;">       </span><span style="font-weight: 400;">2</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2015-12-12</span> <span style="font-weight: 400;">00</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">17</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">39.000</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2015-12-12</span> <span style="font-weight: 400;">01</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">14</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">18.000</span><span style="font-weight: 400;"> |               </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> |         </span><span style="font-weight: 400;">38.48</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">5</span><span style="font-weight: 400;"> | N                  |           </span><span style="font-weight: 400;">68</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">265</span><span style="font-weight: 400;"> |            </span><span style="font-weight: 400;">2</span><span style="font-weight: 400;"> |       </span><span style="font-weight: 400;">200.0</span><span style="font-weight: 400;"> |   </span><span style="font-weight: 400;">0.0</span><span style="font-weight: 400;"> |   </span>

<span style="font-weight: 400;">       </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2016-12-29</span> <span style="font-weight: 400;">07</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">46</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">33.000</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2016-12-29</span> <span style="font-weight: 400;">08</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">03</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">36.000</span><span style="font-weight: 400;"> |               </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> |           </span><span style="font-weight: 400;">4.7</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> | N                  |          </span><span style="font-weight: 400;">151</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">223</span><span style="font-weight: 400;"> |            </span><span style="font-weight: 400;">2</span><span style="font-weight: 400;"> |        </span><span style="font-weight: 400;">16.5</span><span style="font-weight: 400;"> |   </span><span style="font-weight: 400;">0.0</span><span style="font-weight: 400;"> |   </span>

<span style="font-weight: 400;">       </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2014-08-15</span> <span style="font-weight: 400;">10</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">37</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">42.000</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2014-08-15</span> <span style="font-weight: 400;">11</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">07</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">21.000</span><span style="font-weight: 400;"> |               </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">10.0</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> | N                  |          </span><span style="font-weight: 400;">230</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">138</span><span style="font-weight: 400;"> |            </span><span style="font-weight: 400;">3</span><span style="font-weight: 400;"> |        </span><span style="font-weight: 400;">32.0</span><span style="font-weight: 400;"> |   </span><span style="font-weight: 400;">0.0</span><span style="font-weight: 400;"> |   </span>

<span style="font-weight: 400;">       </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2015-01-06</span> <span style="font-weight: 400;">18</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">27</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">38.000</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2015-01-06</span> <span style="font-weight: 400;">19</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">04</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">04.000</span><span style="font-weight: 400;"> |               </span><span style="font-weight: 400;">2</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">18.5</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">2</span><span style="font-weight: 400;"> | N                  |          </span><span style="font-weight: 400;">264</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">264</span><span style="font-weight: 400;"> |            </span><span style="font-weight: 400;">2</span><span style="font-weight: 400;"> |        </span><span style="font-weight: 400;">52.0</span><span style="font-weight: 400;"> |   </span><span style="font-weight: 400;">0.0</span><span style="font-weight: 400;"> |   </span>

<span style="font-weight: 400;">       </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2016-03-14</span> <span style="font-weight: 400;">15</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">16</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">32.000</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2016-03-14</span> <span style="font-weight: 400;">16</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">27</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">11.000</span><span style="font-weight: 400;"> |               </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">18.5</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">2</span><span style="font-weight: 400;"> | Y                  |          </span><span style="font-weight: 400;">132</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">230</span><span style="font-weight: 400;"> |            </span><span style="font-weight: 400;">3</span><span style="font-weight: 400;"> |        </span><span style="font-weight: 400;">52.0</span><span style="font-weight: 400;"> |   </span><span style="font-weight: 400;">0.0</span><span style="font-weight: 400;"> |   </span>

<span style="font-weight: 400;">       </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2016-05-02</span> <span style="font-weight: 400;">09</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">10</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">21.000</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2016-05-02</span> <span style="font-weight: 400;">09</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">58</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">00.000</span><span style="font-weight: 400;"> |               </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">18.0</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">2</span><span style="font-weight: 400;"> | N                  |          </span><span style="font-weight: 400;">229</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">132</span><span style="font-weight: 400;"> |            </span><span style="font-weight: 400;">3</span><span style="font-weight: 400;"> |        </span><span style="font-weight: 400;">52.0</span><span style="font-weight: 400;"> |   </span><span style="font-weight: 400;">0.0</span><span style="font-weight: 400;"> |   </span>

<span style="font-weight: 400;">       </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2017-11-28</span> <span style="font-weight: 400;">23</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">24</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">24.000</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2017-11-29</span> <span style="font-weight: 400;">00</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">09</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">00.000</span><span style="font-weight: 400;"> |               </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> |           </span><span style="font-weight: 400;">6.2</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> | N                  |          </span><span style="font-weight: 400;">246</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">112</span><span style="font-weight: 400;"> |            </span><span style="font-weight: 400;">3</span><span style="font-weight: 400;"> |        </span><span style="font-weight: 400;">26.5</span><span style="font-weight: 400;"> |   </span><span style="font-weight: 400;">0.5</span><span style="font-weight: 400;"> |   </span>

<span style="font-weight: 400;">       </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2014-09-14</span> <span style="font-weight: 400;">12</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">37</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">44.000</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2014-09-14</span> <span style="font-weight: 400;">12</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">49</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">49.000</span><span style="font-weight: 400;"> |               </span><span style="font-weight: 400;">2</span><span style="font-weight: 400;"> |           </span><span style="font-weight: 400;">1.4</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> | N                  |          </span><span style="font-weight: 400;">186</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">170</span><span style="font-weight: 400;"> |            </span><span style="font-weight: 400;">3</span><span style="font-weight: 400;"> |         </span><span style="font-weight: 400;">9.5</span><span style="font-weight: 400;"> |   </span><span style="font-weight: 400;">0.0</span><span style="font-weight: 400;"> |   </span>

<span style="font-weight: 400;">       </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2015-11-20</span> <span style="font-weight: 400;">21</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">35</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">41.000</span><span style="font-weight: 400;"> | </span><span style="font-weight: 400;">2015-11-20</span> <span style="font-weight: 400;">22</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">08</span><span style="font-weight: 400;">:</span><span style="font-weight: 400;">29.000</span><span style="font-weight: 400;"> |               </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> |           </span><span style="font-weight: 400;">8.5</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> | N                  |          </span><span style="font-weight: 400;">264</span><span style="font-weight: 400;"> |          </span><span style="font-weight: 400;">264</span><span style="font-weight: 400;"> |            </span><span style="font-weight: 400;">3</span><span style="font-weight: 400;"> |        </span><span style="font-weight: 400;">30.5</span><span style="font-weight: 400;"> |   </span><span style="font-weight: 400;">0.5</span><span style="font-weight: 400;"> |   </span>

<span style="font-weight: 400;">(</span><span style="font-weight: 400;">10</span> <span style="font-weight: 400;">rows</span><span style="font-weight: 400;">)</span>

<span style="font-weight: 400;">Query </span><span style="font-weight: 400;">20230815</span><span style="font-weight: 400;">_131646_00028_4suvq, FINISHED, </span><span style="font-weight: 400;">1</span><span style="font-weight: 400;"> node</span>

<span style="font-weight: 400;">Splits: </span><span style="font-weight: 400;">353</span><span style="font-weight: 400;"> total, </span><span style="font-weight: 400;">353</span><span style="font-weight: 400;"> done (</span><span style="font-weight: 400;">100.00</span><span style="font-weight: 400;">%)</span>

<span style="font-weight: 400;">2.77</span><span style="font-weight: 400;"> [</span><span style="font-weight: 400;">487</span><span style="font-weight: 400;">M </span><span style="font-weight: 400;">rows</span><span style="font-weight: 400;">, </span><span style="font-weight: 400;">6.36</span><span style="font-weight: 400;">GB] [</span><span style="font-weight: 400;">176</span><span style="font-weight: 400;">M </span><span style="font-weight: 400;">rows</span><span style="font-weight: 400;">/s, </span><span style="font-weight: 400;">2.3</span><span style="font-weight: 400;">GB/s]</span>

_col0

-----------

899294289

(1 row)

Query 20230815_132013_00029_4suvq, FINISHED, 1 node

Splits: 343 total, 343 done (100.00%)

1.04 [899M rows, 3.56MB] [865M rows/s, 3.43MB/s]

trino:default> select * from nyctaxi WHERE tolls_amount > 100 order by tolls_amount desc limit 10;

----------+-------------------------+-------------------------+-----------------+---------------+------------+--------------------+--------------+--------------+--------------+-------------+-------+----

2 | 2017-07-03 17:15:15.000 | 2017-07-03 17:35:57.000 | 1 | 9.74 | 1 | N | 138 | 107 | 2 | 27.5 | 1.0 |

2 | 2015-12-12 00:17:39.000 | 2015-12-12 01:14:18.000 | 1 | 38.48 | 5 | N | 68 | 265 | 2 | 200.0 | 0.0 |

1 | 2016-12-29 07:46:33.000 | 2016-12-29 08:03:36.000 | 1 | 4.7 | 1 | N | 151 | 223 | 2 | 16.5 | 0.0 |

1 | 2014-08-15 10:37:42.000 | 2014-08-15 11:07:21.000 | 1 | 10.0 | 1 | N | 230 | 138 | 3 | 32.0 | 0.0 |

1 | 2015-01-06 18:27:38.000 | 2015-01-06 19:04:04.000 | 2 | 18.5 | 2 | N | 264 | 264 | 2 | 52.0 | 0.0 |

1 | 2016-03-14 15:16:32.000 | 2016-03-14 16:27:11.000 | 1 | 18.5 | 2 | Y | 132 | 230 | 3 | 52.0 | 0.0 |

1 | 2016-05-02 09:10:21.000 | 2016-05-02 09:58:00.000 | 1 | 18.0 | 2 | N | 229 | 132 | 3 | 52.0 | 0.0 |

1 | 2017-11-28 23:24:24.000 | 2017-11-29 00:09:00.000 | 1 | 6.2 | 1 | N | 246 | 112 | 3 | 26.5 | 0.5 |

1 | 2014-09-14 12:37:44.000 | 2014-09-14 12:49:49.000 | 2 | 1.4 | 1 | N | 186 | 170 | 3 | 9.5 | 0.0 |

1 | 2015-11-20 21:35:41.000 | 2015-11-20 22:08:29.000 | 1 | 8.5 | 1 | N | 264 | 264 | 3 | 30.5 | 0.5 |

(10 rows)

Query 20230815_131646_00028_4suvq, FINISHED, 1 node

Splits: 353 total, 353 done (100.00%)

2.77 [487M rows, 6.36GB] [176M rows/s, 2.3GB/s]

Trino S3 via hive-metastore Integration

trino> show schemas from hive;

Schema

——————–

default

information_schema

(2 rows)

Query 20230810_152954_00003_a45jd, FINISHED, 3 nodes

Splits: 68 total, 68 done (100.00%)

0.61 [2 rows, 35B] [3 rows/s, 57B/s]

Upskill Your Knowledge!

Oracle DBaaS Products Comparison

Dremio S3 and NFS Integration

Hive-metastore on Kubernetes with S3 External Table

Airbyte S3 Connector on Kubernetes

Why NAND Flash is Reliable for Enterprise Workloads

4 Data Deduplication Challenges (and How to Solve Them)

SAN vs. NAS vs. DAS: What’s the Difference?

QLC vs. MLC: Which Is the Better SSD?

MongoDB vs. PostgreSQL

Trino S3 via hive-metastore Integration

trino> show schemas from hive;

Schema

——————–

default

information_schema

(2 rows)

Query 20230810_152954_00003_a45jd, FINISHED, 3 nodes

Splits: 68 total, 68 done (100.00%)

0.61 [2 rows, 35B] [3 rows/s, 57B/s]

Upskill Your Knowledge!

Related Stories

Top Stories