Recursive-Tree-Node-Importance

The problem

It is difficult to distinguish which parts of a large collection of documents contain meaningful information versus repetitive noise. Identifying the most important data points manually is time-consuming and often inaccurate.

What it does

It analyzes groups of documents and assigns a score based on how much unique and frequent information they contain. It looks at the structure of the data to highlight which sections carry the most weight.

Why it matters

It provides a clear way to see which parts of a dataset actually contain the most information.

Validation

It was run inside an isolated container with no network access. This is the exact command and the real output it produced — captured process output, not written by a model.

$ python3 recursive_tree_node_importance.py
Root Node Importance: 7.25
Child1 Node Importance: 2.25
Child2 Node Importance: 2.00
Leaf1 Node Importance: 0.00
Leaf2 Node Importance: 0.00

A screenshot of that run.

A clean run proves this does what is shown above, in a CPU-only sandbox. It is a small research demo — not a production tool, and nothing here was published anywhere.

The code

All of it — 83 lines, one file, standard library only.

# Recursive Tree Node Importance Calculator

class Node:
    def __init__(self, name):
        self.name = name
        self.children = []
        self.terms = []
        self.unique_terms = set()
        self.leaf_count = 0
        self.signal_density = 0
        self.importance = 0

    def add_child(self, child):
        self.children.append(child)

    def add_term(self, term):
        self.terms.append(term)
        self.unique_terms.add(term)

    def calculate_importance(self, freq_weight=0.5, div_weight=0.5):
        if not self.children:
            # Leaf node
            self.leaf_count = 1
            if self.terms:
                term_count = len(self.terms)
                unique_terms = len(self.unique_terms)
                diversity = unique_terms / term_count
                frequency = term_count
                self.signal_density = (freq_weight * frequency + div_weight * diversity) * self.leaf_count
            return self.leaf_count, self.signal_density
        
        total_leaf = 0
        total_signal = 0
        for child in self.children:
            child_leaf, child_signal = child.calculate_importance(freq_weight, div_weight)
            total_leaf += child_leaf
            total_signal += child_signal
        
        # Calculate for current node
        self.leaf_count = total_leaf
        if self.terms:
            term_count = len(self.terms)
            unique_terms = len(self.unique_terms)
            diversity = unique_terms / term_count
            frequency = term_count
            node_signal = (freq_weight * frequency + div_weight * diversity) * total_leaf
            self.signal_density = node_signal + total_signal
        else:
            self.signal_density = total_signal
        
        self.importance = self.signal_density
        return total_leaf, self.signal_density

# Example usage
if __name__ == "__main__":
    # Create nodes
    root = Node("root")
    child1 = Node("child1")
    child2 = Node("child2")
    leaf1 = Node("leaf1")
    leaf2 = Node("leaf2")

    root.add_child(child1)
    root.add_child(child2)
    child1.add_child(leaf1)
    child2.add_child(leaf2)

    # Add terms
    root.add_term("common")
    root.add_term("global")
    child1.add_term("specific")
    leaf1.add_term("unique")
    leaf1.add_term("unique")  # Duplicate
    child2.add_term("shared")
    leaf2.add_term("special")

    # Calculate importance
    total_leaves, root_importance = root.calculate_importance()
    print(f"Root Node Importance: {root_importance:.2f}")
    print(f"Child1 Node Importance: {child1.importance:.2f}")
    print(f"Child2 Node Importance: {child2.importance:.2f}")
    print(f"Leaf1 Node Importance: {leaf1.importance:.2f}")
    print(f"Leaf2 Node Importance: {leaf2.importance:.2f}")