Skip to content

P
Postgres FD Implementation
Commit 47309464 authored by Tom Lane's avatar Tom Lane
Browse files
Options

Rewrite GiST documentation into something actually useful. 

Christopher Kings-Lynne
parent 9bd04c01
Show whitespace changes
Inline Side-by-side
Showing with 260 additions and 110 deletions
doc/src/sgml/gist.sgml
View file @ 47309464
<!-- <!--
$Header: /cvsroot/pgsql/doc/src/sgml/gist.sgml,v 1.12 2003/09/29 18:18:35 momjian Exp $ $Header: /cvsroot/pgsql/doc/src/sgml/gist.sgml,v 1.13 2003/10/31 22:41:21 tgl Exp $
--> -->
<Chapter Id="gist"> <chapter Id="GiST">
<DocInfo> <title>GiST Indexes</title>
<AuthorGroup>
<Author> <sect1 id="intro">
<FirstName>Gene</FirstName> <title>Introduction</title>
<Surname>Selkov</Surname>
</Author> <para>
</AuthorGroup> <acronym>GiST</acronym> stands for Generalized Search Tree. It is a
<Date>Transcribed 1998-02-19</Date> balanced, tree-structured access method, that acts as a base template in
</DocInfo> which to implement arbitrary indexing schemes. B+-trees, R-trees and many
<Title>GiST Indexes</Title> other indexing schemes can be implemented in <acronym>GiST</acronym>.
</para>
<Para>
The information about GIST is at <para>
<ULink url="http://GiST.CS.Berkeley.EDU:8000/gist/">http://GiST.CS.Berkeley.EDU:8000/gist/</ULink> One advantage of <acronym>GiST</acronym> is that it allows the development
of custom data types with the appropriate access methods, by
with more on different indexing and sorting schemes at an expert in the domain of the data type, rather than a database expert.
<ULink url="http://s2k-ftp.CS.Berkeley.EDU:8000/personal/jmh/">http://s2k-ftp.CS.Berkeley.EDU:8000/personal/jmh/</ULink>. </para>
And there is more interesting reading at <para>
<ULink url="http://epoch.cs.berkeley.edu:8000/">http://epoch.cs.berkeley.edu:8000/</ULink> and Some of the information here is derived from <ulink
<ULink url="http://www.sai.msu.su/~megera/postgres/gist/">http://www.sai.msu.su/~megera/postgres/gist/</ULink>. url="http://gist.cs.berkeley.edu/">the University of California at
</para> Berkeley's GiST Indexing Project web site</ulink> and Marcel Kornacker's
thesis,
<Para> <ulink url="http://citeseer.nj.nec.com/448594.html">Access Methods for
<Note> Next-Generation Database Systems</ulink>. The <acronym>GiST</acronym>
<Title>Author</Title> implementation in <productname>PostgreSQL</productname> is primarily
<Para> maintained by Teodor Sigaev and Oleg Bartunov, and there is more
This extraction from an email sent by information on their website: <ulink
Eugene Selkov, Jr. (<email>selkovjr@mcs.anl.gov</email>) url="http://www.sai.msu.su/~megera/postgres/gist/"></>.
contains good information </para>
on GiST. Hopefully we will learn more in the future and update this information.
- thomas 1998-03-01 </sect1>
</Para>
</Note> <sect1 id="extensibility">
</para> <title>Extensibility</title>
<Para>
Well, I can't say I quite understand what's going on, but at least <para>
I (almost) succeeded in porting GiST examples to linux. The GiST access Traditionally, implementing a new index access method meant a lot of
method is already in the postgres tree (<FileName>src/backend/access/gist</FileName>). difficult work. It was necessary to understand the inner workings of the
</para> database, such as the lock manager and Write-Ahead Log. The
<Para> <acronym>GiST</acronym> interface has a high level of abstraction,
<ULink url="ftp://s2k-ftp.cs.berkeley.edu/pub/gist/pggist/pggist.tgz">Examples at Berkeley</ULink> requiring the access method implementor to only implement the semantics of
come with an overview of the methods and demonstrate spatial index the data type being accessed. The <acronym>GiST</acronym> layer itself
mechanisms for 2D boxes, polygons, integer intervals and text takes care of concurrency, logging and searching the tree structure.
(see also <ULink url="http://gist.cs.berkeley.edu:8000/gist/">GiST at Berkeley</ULink>). </para>
In the box example, we
are supposed to see a performance gain when using the GiST index; it did <para>
work for me but I do not have a reasonably large collection of boxes This extensibility should not be confused with the extensibility of the
to check that. Other examples also worked, except polygons: I got an other standard search trees in terms of the data they can handle. For
error doing example, <productname>PostgreSQL</productname> supports extensible B+-trees
and R-trees. That means that you can use
<ProgramListing> <productname>PostgreSQL</productname> to build a B+-tree or R-tree over any
test=> CREATE INDEX pix ON polytmp data type you want. But B+-trees only support range predicates
test-> USING GIST (p:box gist_poly_ops) WITH (ISLOSSY); (<literal>&lt;</literal>, <literal>=</literal>, <literal>&gt;</literal>),
ERROR: cannot open pix and R-trees only support n-D range queries (contains, contained, equals).
</para>
(PostgreSQL 6.3 Sun Feb 1 14:57:30 EST 1998)
</ProgramListing> <para>
</para> So if you index, say, an image collection with a
<Para> <productname>PostgreSQL</productname> B+-tree, you can only issue queries
I could not get sense of this error message; it appears to be something such as <quote>is imagex equal to imagey</quote>, <quote>is imagex less
we'd rather ask the developers about (see also Note 4 below). What I than imagey</quote> and <quote>is imagex greater than imagey</quote>?
would suggest here is that someone of you linux guys (linux==gcc?) fetch the Depending on how you define <quote>equals</quote>, <quote>less than</quote>
original sources quoted above and apply my patch (see attachment) and and <quote>greater than</quote> in this context, this could be useful.
tell us what you feel about it. Looks cool to me, but I would not like However, by using a <acronym>GiST</acronym> based index, you could create
to hold it up while there are so many competent people around. ways to ask domain-specific questions, perhaps <quote>find all images of
</para> horses</quote> or <quote>find all over-exposed images</quote>.
<Para> </para>
A few notes on the sources:
</para> <para>
<Para> All it takes to get a <acronym>GiST</acronym> access method up and running
1. I failed to make use of the original (HP-UX) Makefile and rearranged is to implement seven user-defined methods, which define the behavior of
the Makefile from the ancient postgres95 tutorial to do the job. I tried keys in the tree. Of course these methods have to be pretty fancy to
to keep it generic, but I am a very poor makefile writer -- just did support fancy queries, but for all the standard queries (B+-trees,
some monkey work. Sorry about that, but I guess it is now a little R-trees, etc.) they're relatively straightforward. In short,
more portable that the original makefile. <acronym>GiST</acronym> combines extensibility along with generality, code
</para> reuse, and a clean interface.
<Para> </para>
2. I built the example sources right under pgsql/src (just extracted the
tar file there). The aforementioned Makefile assumes it is one level </sect1>
below pgsql/src (in our case, in pgsql/src/pggist).
</para> <sect1 id="implementation">
<Para> <title>Implementation</title>
3. The changes I made to the *.c files were all about #include's,
function prototypes and typecasting. Other than that, I just threw <para>
away a bunch of unused vars and added a couple parentheses to please There are seven methods that an index operator class for
gcc. I hope I did not screw up too much :) <acronym>GiST</acronym> must provide:
</para> </para>
<Para>
4. There is a comment in polyproc.sql: <variablelist>
<varlistentry>
<ProgramListing> <term>consistent</term>
-- -- there's a memory leak in rtree poly_ops!! <listitem>
-- -- CREATE INDEX pix2 ON polytmp USING RTREE (p poly_ops); <para>
</ProgramListing> Given a predicate <literal>p</literal> on a tree page, and a user
query, <literal>q</literal>, this method will return false if it is
Roger that!! I thought it could be related to a number of certain that both <literal>p</literal> and <literal>q</literal> cannot
<ProductName>PostgreSQL</ProductName> versions be true for a given data item.
back and tried the query. My system went nuts and I had to shoot down </para>
the postmaster in about ten minutes. </listitem>
</para> </varlistentry>
<Para> <varlistentry>
I will continue to look into GiST for a while, but I would also <term>union</term>
appreciate <listitem>
more examples of R-tree usage. <para>
</para> This method consolidates information in the tree. Given a set of
</Chapter> entries, this function generates a new predicate that is true for all
the entries.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>compress</term>
<listitem>
<para>
Converts the data item into a format suitable for physical storage in
an index page.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>decompress</term>
<listitem>
<para>
The reverse of the <function>compress</function> method. Converts the
index representation of the data item into a format that can be
manipulated by the database.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>penalty</term>
<listitem>
<para>
Returns a value indicating the <quote>cost</quote> of inserting the new
entry into a particular branch of the tree. items will be inserted
down the path of least <function>penalty</function> in the tree.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>picksplit</term>
<listitem>
<para>
When a page split is necessary, this function decides which entries on
the page are to stay on the old page, and which are to move to the new
page.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>same</term>
<listitem>
<para>
Returns true if two entries are identical, false otherwise.
</para>
</listitem>
</varlistentry>
</variablelist>
</sect1>
<sect1 id="limitations">
<title>Limitations</title>
<para>
The current implementation of <acronym>GiST</acronym> within
<productname>PostgreSQL</productname> has some major limitations:
<acronym>GiST</acronym> access is not concurrent; the
<acronym>GiST</acronym> interface doesn't allow the development of certain
data types, such as digital trees (see papers by Aoki et al); and there
is not yet any support for write-ahead logging of updates in
<acronym>GiST</acronym> indexes.
</para>
<para>
Solutions to the concurrency problems appear in Marcel Kornacker's
thesis; however these ideas have not yet been put into practice in the
<productname>PostgreSQL</productname> implementation.
</para>
<para>
The lack of write-ahead logging is just a small matter of programming,
but since it isn't done yet, a crash could render a <acronym>GiST</acronym>
index inconsistent, forcing a REINDEX.
</para>
</sect1>
<sect1 id="examples">
<title>Examples</title>
<para>
To see example implementations of index methods implemented using
<acronym>GiST</acronym>, examine the following contrib modules:
</para>
<variablelist>
<varlistentry>
<term>btree_gist</term>
<listitem>
<para>B-Tree</para>
</listitem>
</varlistentry>
<varlistentry>
<term>cube</term>
<listitem>
<para>Indexing for multi-dimensional cubes</para>
</listitem>
</varlistentry>
<varlistentry>
<term>intarray</term>
<listitem>
<para>RD-Tree for one-dimensional array of int4 values</para>
</listitem>
</varlistentry>
<varlistentry>
<term>ltree</term>
<listitem>
<para>Indexing for tree-like stuctures</para>
</listitem>
</varlistentry>
<varlistentry>
<term>rtree_gist</term>
<listitem>
<para>R-Tree</para>
</listitem>
</varlistentry>
<varlistentry>
<term>seg</term>
<listitem>
<para>Storage and indexed access for <quote>float ranges</quote></para>
</listitem>
</varlistentry>
<varlistentry>
<term>tsearch and tsearch2</term>
<listitem>
<para>Full text indexing</para>
</listitem>
</varlistentry>
</variablelist>
</sect1>
</chapter>
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment