Posts

Showing posts from January, 2025

[DRAFT] Yugabyte and Postgres - Finding information.

Image
 TL;DR: [DRAFT] Could not resist creating an ERD of tables to collect metrics. I have put "Session" central to my information. Considering it is the application-connect that initiates the session and is the main "customer" causing all activity in a database. Regarding Session: a Datamodel... Aim is currently to find heavy processes and heavy queries.  Spotting underlying tabblets / tables may/should follow from that.  But other may be more interested to dive directly into the storage-layer  (as thew metrics on port 9000 seem to focus on rocksdb) Central, I think, is the "Session". as identified from from pg_stat_activity and from. the ps -ef list on the host. I then built a number of entities (tables) around the Session with the aim of collecting data. Relevant Data (I hope).  Metrics... I did get lost a bit in the huge amout of text coming from ports 7000 and 9000. They seem to provide a lot of rocks-db and program/process info. But I could not (yet) rel...

[Draft] Yugabyte ASH notes and suggestions.

Image
[DRAFT, not for wider publication] TL;DR: Request for adding ID fields to yb_active_session_history. First, use the tablet_uuid instead of the first string-part of the uuid. Furthermore, add IDs to identify the descriptive texts (but keep the text for human readable). And finally consider defining the Root-request-ID as a separate entity (possibly another view: yb_root_request).  Elaborate .. So far I have created a table to store yb_active_session_history records. The columns in the table are copies of the columns from the view, with two additions: id and host. The id comes from  sequence, and the host is the $HOSTNAME of the node where the tserver runs, e.g. where the record is collected (in future, I would use tsrv_uuid). My Ash records are "under development", but this is my latest: I also built a stored-function to capture the records, which was made to run on every node (every tserver) with 1 or 3 or 10min intervals (insert where not exist...). The resulting data can be...

[Draft] Postgres - Yugabyte: logging of sessions, and (active-) session-history

Image
[DRAFT - for discussion only]  TL;DR: After examining the possibilities of pg_stat_... and combining it with yb_active_session_history (ASH), I have some ideas and suggestions to help with finding hotspots and heavy statements. Background: My unstoppable urge to investigate databases via SQL, and a focus on "knowing what happens" with connections and queries. Philosophy: Queries are the root of all (user-)activity and (user-)problems. Queries come from application-connections or sessions. And in a Distributed Database (DD), the sessions and their queries can cause work on any node in the cluster. Hence you need to be able to "track them down".  In PostgreSQL we have pg_stat (_activity, _statements) for that. Yugabyte added a feature called Active Session History (ASH), and I am combining that with information from pg_stat_activity and pg_stat_statements to try and identify "heavy queries" and relate them back to session, and thus to users and applications....