Repository navigation
[WIP] Added notebook to explain the algorithm/math of overlap-add - #43
NimaSarajpoor wants to merge 9 commits into
Conversation
|
Found 1 changed notebook. Review the changes at https://app.gitnotebooks.com/stumpy-dev/sliding_dot_product/pull/43 |
|
@seanlaw |
|
Thanks @NimaSarajpoor. I will take a look |
There was a problem hiding this comment.
I haven't read through the "Option I" in detail but I do like how you're building up the story! It almost feels like we might want to add an "overarching goals"/"questions to be answered" section to the top of the document and explain why the goals/questions exist or why they are important (i.e., how are they related to the software that we are writing?) Then, when we address them, we refer back to the goals/questions and say "and now you see how this is related/solved".
At the end of the day, the document isn't only about the convolution concept. It is also about, say, "where does the 'range' come from and how/why does it change when we use a different approach?". I tend to notice that when a "smart" person has spent a long time standing knee deep in the space, everything about the topic seems obvious to them and the "connections" are clear. However, us "dumb" people need you to "state the obvious" in writing with excruciating detail (i.e., don't skip steps!) and to declare those "connections" out loud and then we will be forever grateful.
Sounds good! I will add something in the beginning to help readers understand the objectives and why they matter.
"If you can't explain it simply, you don't understand it well enough" So, whenever something isn’t clearly elaborated, I take it as a reminder to make sure that not only is the communication clear, but that I actually understand the concept well enough to explain it clearly 😅 |
|
@seanlaw |
|
@seanlaw As a side, I think the efficiency of the overlap-add method requires its own discussion. Still, I tried to add something at the end of the notebook to provide some initial ideas for later. |
|
@NimaSarajpoor I am out of town this week but will try to find some time to review next week |
seanlaw
left a comment
seanlaw
left a comment
There was a problem hiding this comment.
@NimaSarajpoor I skimmed the document but I really, really like how thoroughly you've stepped through everything. The story is very clear (though, I haven't studied the math in detail). Eventually, I think you should be able reference different sections of this document from within your code and it will be a wonderful complementary document!
| "id": "199eb655-fde9-4ee1-82e4-55bad5892f22", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "The paper [\"Matrix Profile I\"](https://www.cs.ucr.edu/~eamonn/PID4481997_extend_Matrix%20Profile_I.pdf) uses the [MASS algorithm](https://www.cs.unm.edu/~mueen/FastestSimilaritySearch.html) to compute the distance between a query $Q$ and every subsequence of length $len(Q)$ in $T$. As part of this algorithm, the sliding dot product (sdp) is calculated. The [MASS paper](https://link.springer.com/epdf/10.1007/s10618-024-01005-2?sharing_token=067pAxnaLDvz89q1n_GJt_e4RwlQNchNByi7wbcMAY6HNwOWuMxQSNE3HcKcuL8siHB8L8krJpchQVaGvicUgoegxxV7BWgaU4Y9evg1FU1LGtxlvM9A5UrxrtkqDHyvoOPk_ttVNps-6-LSRf1vuRgFWJI7qrkqWGitjsXfRsw%3D) proposes different variants of MASS for computing the sliding dot product. All the proposed methods use some form of convolution at their core. This notebook is created to help us understand how convolution comes into the picture for computing the sliding dot product. In particular, this notebook answers the following questions:\n", |
There was a problem hiding this comment.
"All OF the proposed methods..."
| "\n", | ||
| "1. **How is the sliding dot product (sdp) related to linear convolution?** This can help us discover the fundamental relationship between sdp and linear convolution\n", | ||
| "2. **How can we compute the convolution faster?** Convolution is a well-studied area. The goal is to review the existing methods to eventually compute sdp faster\n", | ||
| "3. **How to compute convolution faster when $T$ is very long?** This is where we learn about overlap-add method, a divide-and-conquer approach for convolution" |
There was a problem hiding this comment.
"How CAN WE compute convolution faster..."
There was a problem hiding this comment.
"about THE overlap-add method..."
| "* min(idx) is coming from the lower bound of interval $[\\,i,\\ i+m-1\\,]$ when $i=0$. This gives: $min(idx)=0$\n", | ||
| "* max (idx) is coming from the upper bound of interval $[\\,i,\\ i+m-1\\,]$ when $i=n-1$. This gives: $max(idx) = (n-1)+m-1 = n+m-2$\n", | ||
| "\n", | ||
| "When $idx$ changes from $0$ to $n+m-2$, there **exist at least one $i$** such that both $T[i]$ and $Q'[idx-i]$ can be non-zero values. This mathematically shows that the linear convolution $C$ can have values at indices $\\set{0, 1, ..., n + m - 2}$. The length of linear convolution is $n + m - 1$." |
There was a problem hiding this comment.
"The length of THE linear convolution..."
| "\n", | ||
| "> When the DFT is computed for purely real input, the output is Hermitian-symmetric, i.e., the negative frequency terms are just the complex conjugates of the corresponding positive-frequency terms.\n", | ||
| "\n", | ||
| "RFFT/IRFFT is a certain type of FFT/IFFT that leverages this property to perform faster fourier transform. It turns out that the FFT and IFFT in Eq. (18) can be replaced with RFFT and RFFT when inputs are real-valued arrays. This should improve the performance of computing the circular convolution when input sequences have only real values." |
There was a problem hiding this comment.
Is there a typo? Should it say "Eq. (18) can be replaced with RFFT and IRFFT"?
Also, I am not seeing equation numbers :(
2 participants
[WIP] This PR provides a notebook that explains the math behind overlap-add in convolution.